批量去重URL地址并剔除打不开网址

最新推荐文章于 2024-10-25 16:16:17 发布

转载最新推荐文章于 2024-10-25 16:16:17 发布 · 159 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/xiaobaichuangtianxia/p/3842562.html

本文介绍了一个使用Python编写的脚本，该脚本能够批量读取主机列表，并检查每个主机的Web服务状态。对于状态正常的主机，脚本进一步获取其DNS解析地址，并将结果汇总输出到文件中。此工具适用于快速检测多个网站的服务可用性和DNS配置情况。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

#coding=utf-8

import os
import httplib
import socket

dictlist ={};

def ReadHost():
    hosts = [];
    obn = open('d:/sss.txt', 'rb');
    for line in obn:
        #sometime you should filter \r\n
        line = line.strip('\n')
        hosts.append(line)
    obn.close();
    return hosts;

def GetWebStatus(host):
    try:
        conn = httplib.HTTPConnection(host)
        conn.request('GET','url')
        result = conn.getresponse()
        resultStatus= result.status
       # print(host,resultStatus)
        conn.close()
        if(resultStatus != 200):
            return 0
        else:
            return 1
    except httplib.HTTPException,e:
        return 0

def SysDNS():
    hosts = ReadHost();

    for host in hosts:
        #print(host)
        try:

            if (GetWebStatus(host) == 0):
                continue
            myaddrs = socket.getaddrinfo(host,None)
            for eachaddr in myaddrs:
                addrs = eachaddr[4][0]
                #print((addrs))
                if(dictlist.has_key(addrs)):
                    break;
                else:
                    dictlist[addrs] = host;
                    #print(host)
                    break;
        except socket.herror,e:
            continue;
        except socket.gaierror,e1:
            continue;
        except Exception as e2:
            print(e2)
            continue



def showDict():
    fw = open("d:/out.txt","wb");
    for (k,v) in dictlist.items():
        #print(k,v)
        fw.writelines(v);
    fw.close();

if __name__ == "__main__":
    SysDNS();
    showDict();

url可能会出现的错误:

[Errno 10060]
[Errno 10061]
[Errno 10054]
[Errno 10053]

转载于:https://www.cnblogs.com/xiaobaichuangtianxia/p/3842562.html