解决：javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure 问题解决方案

最新推荐文章于 2025-04-21 18:09:30 发布

Adversity-sl

最新推荐文章于 2025-04-21 18:09:30 发布

阅读量5.5k

点赞数

文章标签： ssl 爬虫 java

本文链接：https://blog.youkuaiyun.com/qq_40821163/article/details/120799117

版权

本文档介绍了在使用JavaWebMagic爬虫时遇到的证书错误问题，提供了解决方案，包括重写HttpClientGenerator和HttpClientDownloader两个类，通过忽略SSL证书验证来确保爬虫正常运行。此外，还分享了如何根据目标网站的协议设置SSLContext，并给出了创建Spider时调用新方法的示例代码。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

用java WebMagic爬虫中创建spider过程中出现以下错误：
先贴错误代码：

解决方案：

重写HttpClientGenerator和HttpClientDownloader两个方法

重写HttpClientGenerator方法：

package com.spider.task;

import org.apache.http.HttpException;
import org.apache.http.HttpRequest;
import org.apache.http.HttpRequestInterceptor;
import org.apache.http.client.CookieStore;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.config.SocketConfig;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.DefaultHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.impl.client.*;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.impl.cookie.BasicClientCookie;
import org.apache.http.protocol.HttpContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import us.codecraft.webmagic.Site;
import us.codecraft.webmagic.downloader.CustomRedirectStrategy;

import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import java.io.IOException;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import java.util.Map;

/**
 * @author 咸鱼
 * @date 2018/12/28 21:19
 */
public class HttpClientGenerator {
   

    private transient Logger logger = LoggerFactory.getLogger(getClass());

    private PoolingHttpClientConnectionManager connectionManager;

    public HttpClientGenerator() {
   
        Registry<ConnectionSocketFactory> reg = RegistryBuilder.<ConnectionSocketFactory>create()
                .register("http", PlainConnectionSocketFactory.INSTANCE)
                .register("https", buildSSLConnectionSocketFactory())
                .build();
        connectionManager = new PoolingHttpClientConnectionManager(reg);
        connectionManager.setDefaultMaxPerRoute(100);
    }

    private SSLConnectionSocketFactory buildSSLConnectionSocketFactory() {
   
        try {
   
            // 优先绕过安全证书
            return new SSLConnectionSocketFactory(createIgnoreVerifySSL(), new String[]{
   "SSLv3", "TLSv1", "TLSv1.1", "TLSv1.2"},
                    null,
                    new DefaultHostnameVerifier());
        } catch (KeyManagementException e) {
   
            logg

最低0.47元/天解锁文章