ElasticSearch 分页查询scroll超过10万条数据的方法

最新推荐文章于 2024-07-12 12:16:53 发布

MarvelHonor

最新推荐文章于 2024-07-12 12:16:53 发布

阅读量275

点赞数 2

分类专栏： IT运维 python Linux 文章标签： elasticsearch android 大数据

本文链接：https://blog.youkuaiyun.com/chscomfaner/article/details/138648881

版权

def fetch_1day_user():
    data=[]
    must = []
    now = time.strftime('%Y-%m-%dT%H:%M:%S+0800', time.localtime())
    prior = time.localtime(time.time()-15*60*60)
    prior = time.strftime('%Y-%m-%dT%H:%M:%S+0800', prior)
    # prior = time.strftime('%Y-%m-%dT00:00:00+0800', time.localtime())
    # prior_ = time.strptime(prior,"%Y-%m-%dT%H:%M:%S+0800")
    # now_ = time.mktime(prior_)+24*60*60
    # now = time.strftime('%Y-%m-%dT%H:%M:%S+0800', time.localtime(now_))
    # must.append({"term": {"site": config["gc"]["site"]}})
    must.append({"range": {"@timestamp": {"gt": prior, "lt": now}}})
    must.append({"exists": {"field":"winlog.event_data.TargetUserName.keyword"}})
    # must.append({"regexp": {"winlog.event_data.TargetUserName.keyword": ".*&#