HttpClient 爬接口报 Input length = 1

最新推荐文章于 2024-08-22 10:28:52 发布

破敌逆袭

最新推荐文章于 2024-08-22 10:28:52 发布

阅读量491

点赞数

本文链接：https://blog.youkuaiyun.com/MaYaHei/article/details/94588268

版权

HttpClient 专栏收录该内容

0 篇文章

订阅专栏

使用HttpClient爬取接口时遇到`Input length = 1`错误，由于响应Header中`Set-Cookie`字段含有不合法字符，导致编码问题。尝试从utf-8更改为gbk、gb2312后解决了MalformedInputException，但随后又出现Invalid cookie header，原因是不正确的expires格式。解决方法是自定义cookieSpec，提供相关代码以供参考。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

HttpClient 爬接口报 Input length = 1

调试时发现响应的header有条
Set-Cookie: security_session_verify=cad9721cc133dd9d4646bb8913a27e44; expires=��, 28- 6��-19 15:43:50 GMT; path=/; HttpOnly;
当时初始化的HttpClient编码为utf-8,

  ConnectionConfig connCfg		= ConnectionConfig.custom().setCharset(Charset.forName("utf-8")).build();																
  HttpClientBuilder clientBuilder	= HttpClients.custom()
    										.setConnectionManager(connManager)
    										.setDefaultRequestConfig(defaultRequestConfig)
    										.setDefaultConnectionConfig(connCfg);

包含不合法字符导致读头的时候报java.nio.charset.MalformedInputException: Input length = 1
最后尝试下改换gbk,gb2312,终于不报length=1了.
本以为解决了,结果调试发现cookie 解析会报Invalid cookie header导致丢弃,原来解决乱码后expires=四, 27- 6月-19 20:57:11 GMT,这其实也不是正确的expires格式.
这个时候就得重写cookieSpec了,贴上代码供参考

public class MyCookieSpec extends DefaultCookieSpec {
    public final static String EXPIRESCOOKIESPEC = "myCookieSpec";

    @Override
    public List parse(Header header, CookieOrigin cookieOrigin)
            throws MalformedCookieException
    {

        String value = header.getValue();
        String prefix = "expires=";

        if (value.contains(prefix))
        {
            String expires = value.substring(value.indexOf(prefix) + prefix.length());

            expires = expires.substring(0, expires.indexOf(";"));

            int longer = expires.length();
            //去掉expires
            /*
            if(("?".contains(expires)||isMessy(expires))&&(longer==25||longer==27)){
                //四, 27- 6月-19 20:57:11 GMT
                String date =
                        DateUtils.formatDate(new Date(System.currentTimeMillis()+3*24*3600*1000),"EEE, dd-MMM-yy HH:mm:ss z");

                value = value.replaceAll(prefix + ".{"+longer+"};",prefix + date + ";");
            }
            */
            value = value.replaceAll(prefix + ".{"+longer+"};","");
        }
        header =new BasicHeader(header.getName(), value);

        return super.parse(header, cookieOrigin);

    }
    
   HttpClientContext  httpContext = HttpClientContext.create();
   Registry<CookieSpecProvider> registry = RegistryBuilder.<CookieSpecProvider>create()
                    .register(CookieSpecs.DEFAULT, new DefaultCookieSpecProvider())
                    .register(MyCookieSpec.EXPIRESCOOKIESPEC, context -> new MyCookieSpec())
                    .build();
   httpContext .setCookieSpecRegistry(registry);