Jsoup爬取缺少安全证书异常

最新推荐文章于 2024-09-09 12:36:40 发布

清风凌冽

最新推荐文章于 2024-09-09 12:36:40 发布

阅读量878

点赞数 2

CC 4.0 BY-SA版权

分类专栏： java爬虫之Jsoup 文章标签： java bug ssl https

本文链接：https://blog.youkuaiyun.com/qq_42897733/article/details/117029900

java爬虫之Jsoup 专栏收录该内容

2 篇文章

订阅专栏

本文介绍了一种在抓取HTTPS网站数据时遇到的安全证书异常及其解决方案。通过自定义信任管理器和主机名验证器，可以忽略SSL证书的有效性检查，从而避免因证书问题导致的数据抓取失败。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

项目场景：

抓取https网站数据失败

问题描述：

出现异常： javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

原因分析：

此异常为缺少安全证书的异常

解决方案：

新增工具类：

/**
 * @ClassName SslUtils
 * @Description TODO
 * @Author 清风
 * @Date
 * @Version 1.0
 **/
public class SslUtils {
    private static void trustAllHttpsCertificates() throws Exception {
        TrustManager[] trustAllCerts = new TrustManager[1];
        TrustManager tm = new miTM();
        trustAllCerts[0] = tm;
        SSLContext sc = SSLContext.getInstance("SSL");
        sc.init(null, trustAllCerts, null);
        HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
    }

    /**
     * 忽略HTTPS请求的SSL证书，必须在openConnection之前调用
     *
     * @throws Exception
     */
    public static void ignoreSsl() throws Exception {
        HostnameVerifier hv = new HostnameVerifier() {
            public boolean verify(String urlHostName, SSLSession session) {
                System.out.println("Warning: URL Host: " + urlHostName + " vs. " + session.getPeerHost());
                return true;
            }
        };
        trustAllHttpsCertificates();
        HttpsURLConnection.setDefaultHostnameVerifier(hv);
    }

    static class miTM implements TrustManager, X509TrustManager {
        public X509Certificate[] getAcceptedIssuers() {
            return null;
        }

        public boolean isServerTrusted(X509Certificate[] certs) {
            return true;
        }

        public boolean isClientTrusted(X509Certificate[] certs) {
            return true;
        }

        public void checkServerTrusted(X509Certificate[] certs, String authType)
                throws CertificateException {
            return;
        }

        public void checkClientTrusted(X509Certificate[] certs, String authType)
                throws CertificateException {
            return;
        }
    }

}