Python Hacking Tools - Web Scraper

最新推荐文章于 2024-09-10 10:10:13 发布

weixin_30696427

最新推荐文章于 2024-09-10 10:10:13 发布

阅读量186

点赞数

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/keepmoving1113/p/11312429.html

本文介绍如何使用Python的Requests和BeautifulSoup库从免费代理网站抓取代理列表。通过安装必要的库并编写简单的Python脚本，可以自动获取代理服务器的IP地址和端口。此教程适合初学者了解Python网络爬虫的基本操作。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Preparation:

Python Libray in the following programming:

1. Requests Document: https://2.python-requests.org//en/master/

2. Beautiful Soup Documentation: https://www.crummy.com/software/BeautifulSoup/bs4/doc/

Install the lib on Kali Linux:

apt-get install python-requests

apt-get install python-bs4

Proxy Domain:

https://free-proxy-list.net/

https://free-proxy-list.com/

Python Scraper Code:

import requests
from bs4 import BeautifulSoup

proxyDomain = "https://free-proxy-list.com/"

r = requests.get(proxyDomain,headers={"User-Agent": "XY"})

soup = BeautifulSoup(r.content, 'html.parser')

table = soup.find('table', {"class" : "table table-striped proxy-list"})

for row in table.find_all('tr'):
    columns = row.find_all('td')
    try:
        print "%s:%s\t%-20s\t%-10s" %(columns[0].get_text(),columns[1].get_text().strip(),columns[2].get_text().strip(),columns[3].get_text().strip())
    except:
        pass