【Java - 框架 - HttpClient】(01) 使用“HttpClient“爬取网页的代码示例 - 快速上手

最新推荐文章于 2025-05-07 13:38:15 发布
最新推荐文章于 2025-05-07 13:38:15 发布
阅读量574 收藏 9
点赞数 16
文章标签： java 爬虫 HttpClient
SUNxRUN
本文链接：https://blog.youkuaiyun.com/HIGK_365/article/details/136965321
版权

使用"HttpClient"爬取网页的代码示例 - 快速上手；

依赖

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpcore</artifactId>
    <version>4.4.14</version>
</dependency>
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.13</version>
</dependency>

示例

使用JDK原生API发送HTTP请求

文件01

HttpURLConnectionDemo

代码01

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.nio.charset.StandardCharsets;

/**
 * 使用JDK原生的API来请求网页
 */
public class HttpURLConnectionDemo {
    public static void main(String[] args) throws IOException {
        String urlStr = "https://www.baidu.com/";
        URL url = new URL(urlStr);
        URLConnection urlConnection = url.openConnection();
        HttpURLConnection httpURLConnection = (HttpURLConnection) urlConnection;
        /*
            请求行
            空格
            请求头
            请求体
         */
        // 设置请求类型
        httpURLConnection.setRequestMethod("GET");
        // 设置请求头
        httpURLConnection.setRequestProperty("Accept-Charset", "utf-8");

        // 获取httpURLConnection的输入流
        try (
                InputStream is = httpURLConnection.getInputStream();
                InputStreamReader isr = new InputStreamReader(is, StandardCharsets.UTF_8);
                BufferedReader br = new BufferedReader(isr);
        ) {
            String line;
            while ((line = br.readLine()) != null) {
                System.out.println(line);
            }
        }
    }
}

结果01

在这里插入图片描述

使用HttpClient发送"无参GET"请求

文件01

HttpClientDemoA

代码01

import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

import java.io.IOException;
import java.nio.charset.StandardCharsets;

/**
 * 使用HttpClient发送无参GET请求
 */
public class HttpClientDemoA {
    public static void main(String[] args) {
        // 可关闭的httpClient客户端，相当于你打开的一个浏览器
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        String urlStr = "https://www.baidu.com/";
        // 构造httpGet请求对象
        HttpGet httpGet = new HttpGet(urlStr);
        /*加请求头的操作*/
        // 解决httpClient被认为不是真人的问题
        httpGet.addHeader("User-Agent:", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.47");
        // 解决防盗链，value：发生防盗链的网址的url
        httpGet.addHeader("Referer", "https://www.baidu.com/");
        // 可关闭的响应
        CloseableHttpResponse response = null;
        try {
            response = closeableHttpClient.execute(httpGet);
            // 获取响应结果：DecompressingEntity
            /*
            HttpEntity不仅可以作为结果，也可以作为请求的参数实体，有很多的实现。
             */
            HttpEntity entity = response.getEntity();
            // 对"HttpEntity"操作的工具类
            String toStringResult = EntityUtils.toString(entity, StandardCharsets.UTF_8);
            System.out.println(toStringResult);
            // 确保流关闭
            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (closeableHttpClient != null) {
                try {
                    closeableHttpClient.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            if (response != null) {
                try {
                    response.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

结果01

在这里插入图片描述

使用HttpClient发送"有参GET"请求

文件01

HttpClientDemoB

代码01

import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;

/**
 * 使用HttpClient发送有参GET请求
 */
public class HttpClientDemoB {
    public static void main(String[] args) throws UnsupportedEncodingException {
        // 可关闭的httpClient客户端，相当于你打开的一个浏览器
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();

        String passwordParam = "123+abc 456|789";// 123 abc
        // 做urlEncode：如果是浏览器的话，是自动做了的，123%2Babc+456%7C789
        URLEncoder.encode(passwordParam, StandardCharsets.UTF_8.name());
        String urlStr = "https://www.baidu.com/";
        // 构造httpGet请求对象
        HttpGet httpGet = new HttpGet(urlStr);

        // 加请求头的操作
        // 解决httpClient被认为不是真人的问题
        httpGet.addHeader("User-Agent:", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.47");
        // 解决防盗链，value：发生防盗链的网址的url
        httpGet.addHeader("Referer", "https://www.baidu.com/");
        // 可关闭的响应
        CloseableHttpResponse response = null;
        try {
            response = closeableHttpClient.execute(httpGet);
            // 获取响应结果：DecompressingEntity
            /*
            HttpEntity不仅可以作为结果，也可以作为请求的参数实体，有很多的实现。
             */
            HttpEntity entity = response.getEntity();
            // 对"HttpEntity"操作的工具类
            String toStringResult = EntityUtils.toString(entity, StandardCharsets.UTF_8);
            System.out.println(toStringResult);
            // 确保流关闭
            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (closeableHttpClient != null) {
                try {
                    closeableHttpClient.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            if (response != null) {
                try {
                    response.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

保存网络上的图片到本地

文件01

GetPicture

代码01

import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

import java.io.FileOutputStream;
import java.io.IOException;

/**
 * 保存网络图片到本地
 */
public class GetPicture {
    public static void main(String[] args) {
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        String urlStr = "https://www.baidu.com/img/pc_675fe66eab33abff35a2669768c43d95.png";
        HttpGet httpGet = new HttpGet(urlStr);

        httpGet.addHeader("User-Agent:", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.47");
        // 可关闭的响应
        CloseableHttpResponse response = null;
        try {
            response = closeableHttpClient.execute(httpGet);
            HttpEntity entity = response.getEntity();
            // image/图片的后缀：image/jpg、image/jpeg、image/png
            String contentType = entity.getContentType().getValue();
            String suffix = ".jpg";
            if (contentType.contains("jpg") || contentType.contains("jpeg")) {
                suffix = ".jpg";
            } else if (contentType.contains("bmp") || contentType.contains("bitmap")) {
                suffix = ".bmp";
            } else if (contentType.contains("png")) {
                suffix = ".png";
            } else if (contentType.contains(".gif")) {
                suffix = ".gif";
            }
            //获取文件的字节流
            byte[] bytes = EntityUtils.toByteArray(entity);
            String localAbsPath = "e:\\abc" + suffix;
            FileOutputStream fos = new FileOutputStream(localAbsPath);
            fos.write(bytes);
            fos.close();
            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (closeableHttpClient != null) {
                try {
                    closeableHttpClient.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

结果01

在这里插入图片描述

设置代理实现访问

文件01

HttpClientDemoC

代码01

import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

import java.io.IOException;
import java.nio.charset.StandardCharsets;

/**
 * 设置访问代理
 */
public class HttpClientDemoC {
    public static void main(String[] args) {
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        String urlStr = "https://www.baidu.com/";
        HttpGet httpGet = new HttpGet(urlStr);
        // 创建一个代理
        String ip = "221.5.80.66";
        int port = 3128;
        HttpHost proxy = new HttpHost(ip, port);
        // 对每一个请求进行配置，会覆盖全局的默认请求配置
        RequestConfig requestConfig = RequestConfig.custom()
            .setProxy(proxy).build();

        httpGet.setConfig(requestConfig);
        CloseableHttpResponse response = null;
        try {
            response = closeableHttpClient.execute(httpGet);
            HttpEntity entity = response.getEntity();

            String toStringResult = EntityUtils.toString(entity, StandardCharsets.UTF_8);
            System.out.println(toStringResult);

            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (closeableHttpClient != null) {
                try {
                    closeableHttpClient.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

设置连接超时和请求超时

文件01

HttpClientDemoD

代码01

import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

import java.io.IOException;
import java.nio.charset.StandardCharsets;

/**
 * 连接超时和读取超时
 */
public class HttpClientDemoD {
    public static void main(String[] args) {
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        String urlStr = "https://www.baidu.com/";
        HttpGet httpGet = new HttpGet(urlStr);
        // 对每一个请求进行配置，会覆盖全局的默认请求配置
        RequestConfig requestConfig = RequestConfig.custom()
                // 设置连接超时,单位:ms,完成TCP三次握手的时间上限
                .setConnectTimeout(5000)
                // 设置读取超时,单位:ms,从请求的网值获得响应数据的时间间隔
                .setSocketTimeout(3000)
                // 指从连接池里面获取connection的超时时间
//                .setConnectionRequestTimeout(1)
                .setConnectionRequestTimeout(5000)
                .build();

        httpGet.setConfig(requestConfig);
        CloseableHttpResponse response = null;
        try {
            response = closeableHttpClient.execute(httpGet);
            HttpEntity entity = response.getEntity();

            String toStringResult = EntityUtils.toString(entity, StandardCharsets.UTF_8);
            System.out.println(toStringResult);

            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (closeableHttpClient != null) {
                try {
                    closeableHttpClient.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

结果01

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

使用HttpClient发送"表单类型的POST"请求

文件01

HttpClientDemoE

代码01

import org.apache.http.HttpEntity;
import org.apache.http.NameValuePair;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;

/**
 * 发送表单类型的POST请求
 * "application/x-www-form-urlencoded"
 */
public class HttpClientDemoE {
    public static void main(String[] args) {
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        String urlStr = "https://www.baidu.com/";
        // 创建httpPost对象
        HttpPost httpPost = new HttpPost(urlStr);
        // 设置请求头
        httpPost.addHeader("Content-Type","application/x-xxx-form-urlencoded; charset=UTF-8");
        // 给post对象设置参数
        /*
          NameValuePair: <input id="user-name-label" type="text" name="userName"/>
          的"name(userName)"和"input"标签里面输入的值就构成了一个NameValuePair对象
        */
        List<NameValuePair> list = new ArrayList<>();
        list.add(new BasicNameValuePair("userName","乐知者java"));
        list.add(new BasicNameValuePair("password","123456"));
        // 把参数集合设置到formEntity
        UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(null,StandardCharsets.UTF_8);
        httpPost.setEntity(formEntity);

        CloseableHttpResponse response = null;
        try {
            response = closeableHttpClient.execute(httpPost);
            HttpEntity entity = response.getEntity();

            String toStringResult = EntityUtils.toString(entity, StandardCharsets.UTF_8);
            System.out.println(toStringResult);

            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (closeableHttpClient != null) {
                try {
                    closeableHttpClient.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

使用HttpClient发送"Json类型的POST"请求

文件01

HttpClientDemoF

代码01

import org.apache.http.Consts;
import org.apache.http.HttpEntity;
import org.apache.http.NameValuePair;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.BasicHttpEntity;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;

/**
 * 发送表单类型的POST请求
 * "application/json"
 */
public class HttpClientDemoF {
    public static void main(String[] args) {
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        String urlStr = "https://www.baidu.com/";
        // 创建httpPost对象
        HttpPost httpPost = new HttpPost(urlStr);

        //String:是一个Json字符串
        JSONObject jsonObj = new JSONObject();
        jsonObj.put("username", "java乐之者");
        jsonObj.put("password", "123456");

        StringEntity jsonEntity = new StringEntity(jsonObj.toString(), Consts.UTF_8);
        // 也需要给entity设置一下内容类型
        jsonEntity.setContentType(new BasicHeader("Content-Type", "application/json; charset=UTF-8"));
        // 设置entity的编码
        jsonEntity.setContentEncoding(Consts.UTF_8.name());
        httpPost.setEntity(jsonEntity);

        CloseableHttpResponse response = null;
        try {
            response = closeableHttpClient.execute(httpPost);
            HttpEntity entity = response.getEntity();

            String toStringResult = EntityUtils.toString(entity, StandardCharsets.UTF_8);
            System.out.println(toStringResult);

            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (closeableHttpClient != null) {
                try {
                    closeableHttpClient.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

使用HttpClient发送"上传文件的POST"请求

文件01

HttpClientDemoG

代码01

import org.apache.http.Consts;
import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.StringEntity;
import org.apache.http.entity.mime.HttpMultipartMode;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.entity.mime.content.StringBody;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.util.EntityUtils;

import java.io.File;
import java.io.IOException;
import java.nio.charset.StandardCharsets;

/**
 * 发送上传文件的POST请求
 * "multipart/from-data"
 */
public class HttpClientDemoG {
    public static void main(String[] args) {
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        String urlStr = "https://www.baidu.com/";
        // 创建httpPost对象
        HttpPost httpPost = new HttpPost(urlStr);
        // 构造一个ContentBody的实现类对象
        FileBody fileBody = new FileBody(new File("e:\\a.png"));
        // 设置构造上传文件使用的entity
        MultipartEntityBuilder builder = MultipartEntityBuilder.create();
        builder.setCharset(Consts.UTF_8);// 设置编码
        builder.setContentType(ContentType.create(
                "multipart/fom-data",Consts.UTF_8));
        builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);// 设置浏览器模式
        // 对于普通的表单字段如果含有中文的话，不能通过addTextBody，否则乱码
        // text：指的是输入的值
        StringBody userNameTextBody = new StringBody("小明",
                ContentType.create(
                        "text/plain",Consts.UTF_8));

        HttpEntity httpEntity = builder.addPart("fileName", fileBody)
                // 通过file,byte[],inputstream来上传文件
                .addBinaryBody("fileName", new File("e:\\b.png"))
                .addPart("username", userNameTextBody)
                .addTextBody("password", "123456")
                .build();

        httpPost.setEntity(httpEntity);
        CloseableHttpResponse response = null;
        try {
            response = closeableHttpClient.execute(httpPost);
            HttpEntity entity = response.getEntity();

            String toStringResult = EntityUtils.toString(entity, StandardCharsets.UTF_8);
            System.out.println(toStringResult);

            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (closeableHttpClient != null) {
                try {
                    closeableHttpClient.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

配置HttpClient绕过"https"安全认证

文件01

HttpClientDemoH

代码01

import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.ssl.TrustStrategy;
import org.apache.http.util.EntityUtils;

import javax.net.ssl.SSLContext;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

/**
 * 配置httpClient绕过https安全认证
 */
public class HttpClientDemoH {

    //创建支持安全协议的连接工厂
    private ConnectionSocketFactory trustHttpsCertificates() throws Exception {
        SSLContextBuilder sslContextBuilder = new SSLContextBuilder();
        //
        sslContextBuilder.loadTrustMaterial(null, new TrustStrategy() {
            // 判断是否信任URL
            @Override
            public boolean isTrusted(X509Certificate[] x509Certificates, String s) throws CertificateException {
                return true;
            }
        });
        SSLContext sslContext = sslContextBuilder.build();

        SSLConnectionSocketFactory sslConnectionSocketFactory = new
                SSLConnectionSocketFactory(sslContext,
                new String[]{"SSLv2Hello", "SSLv3", "TLSv1", "TLSv1.1", "TLSv1.2"}
                , null, NoopHostnameVerifier.INSTANCE);
        return sslConnectionSocketFactory;
    }

    public void start() throws Exception {
        Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
                .register("http", PlainConnectionSocketFactory.INSTANCE)
                .register("https", trustHttpsCertificates())
                .build();
        // 创建一个ConnectionManager
        PoolingHttpClientConnectionManager pool = new PoolingHttpClientConnectionManager(registry);
        // 定制CloseableHttpClient对象
        HttpClientBuilder httpClientBuilder = HttpClients.custom().setConnectionManager(pool);
        // 配置好httpClient之后，通过build方法来获取httpClient对象
        CloseableHttpClient closeableHttpClient = httpClientBuilder.build();
        //  CloseableHttpClient closeableHttpClient = httpClientBuilder.createDefault();
        String urlStr = "https://zcdsade.cfd//";
        HttpGet httpGet = new HttpGet(urlStr);
        CloseableHttpResponse response = null;
        try {
            response = closeableHttpClient.execute(httpGet);
            HttpEntity entity = response.getEntity();
            String toStringResult = EntityUtils.toString(entity, StandardCharsets.UTF_8);
            System.out.println(toStringResult);
            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (closeableHttpClient != null) {
                try {
                    closeableHttpClient.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            if (response != null) {
                try {
                    response.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }

    public static void main(String[] args) throws Exception {
        HttpClientDemoH httpClientDemo = new HttpClientDemoH();
        httpClientDemo.start();
    }
}

【功能实现】

[获取响应头以及相应的"Content-Type"]

sdf

CloseableHttpResponse response = closeableHttpClient.execute(httpGet);

aa

StatusLine statusLine = response.getStatusLine();

sdf

// 代表本次请求的成功、失败的状态
statusLine.getStatusCode;

afa

// 获取所有的响应头
Header[] allHeaders = response.getAllHeaders();
for (Header header : allHeaders) {
System.out.println("响应头" + header.getName() + "的值:" + header.getValue());
}

ewwsr

// 获取响应头"Content-Type"的值
HttpEntity entity = response.getEntity();
System.out.println("ContentType的值:" + entity.getContentType());
httpclient源码
04-19
httpclient的源代码。。 http://www.apache.org
java http客户端源码_Java 配置httpclient
weixin_32572673的博客
02-24 155
[java]代码库public class ConnectionManager {static final int TIMEOUT = 20000;//连接超时时间static final int SO_TIMEOUT = 60000;//数据传输超时public static DefaultHttpClient getHttpClient(){SchemeRegistry schemeRegis...
httpclient4.2.1最新版连接池配置
kidbei
10-09 253
[code="java"] public class ConnectionManager { static final int TIMEOUT = 20000;//连接超时时间 static final int SO_TIMEOUT = 60000;//数据传输超时 public static DefaultHttpClient getHttpClient(){ ...
java http客户端源码_Java 使用 HttpClient调用https 最新源码 JDK7+ apache4.3+
weixin_29063681的博客
02-24 283
在项目使用https方式调用别人服务的时候，以前要写很多的代码，现在框架封装了很多，所以不用再写那么多了。网上看了一下，都是很老的版本，用过时的DefaultHttpClient。以spring为例：1，在apache 4.0版本中有DefaultHttpClient 这个类，是用来做httpsClient的，但是在4.3的时候就换成了CloseableHttpResponse 。Defaul...
完整的HttpClient的get和post使用方法实例（java源代码）
01-18
使用HttpClient的get和post方法实例，Java代码完整，浅显易懂，可执行，基于httpclient4和jgroup。
WebMagic是一个简单灵活的Java爬虫框架
08-08
在实际使用中，WebMagic提供了丰富的示例和文档，帮助开发者快速上手。例如，通过简单的几行代码，就能实现一个基本的爬虫，如下所示： ```java Spider.create(new MyPageProcessor()).addUrl("http://example.com"...
Java爬虫demo：快速实现网站内容爬取
在本份文件中，"爬虫javaDemo" 意味着这是一份用Java编写的爬虫示例程序，它为用户提供了直接可用的代码示例，能够帮助用户实现各种网站内容的抓取。同时，描述中提到该程序易于镶入已有项目，并且简单实用，这表明...
jvppeteer-master.zip
06-12
项目可能包含了数据处理的示例，如使用Java的流API进行数据清洗，以及将数据存入数据库的示例代码。 7. **学习路径**: 对于初学者，可以从了解Puppeteer的基本概念开始，然后深入学习JVPPeteer的API，结合提供的...
Java爬虫框架MelodySpider的介绍与应用
- 易于使用和扩展：框架可能提供清晰的API文档和示例代码，便于开发者快速上手，并且具有良好的模块化设计，方便用户根据自身需求进行功能的拓展和定制。 知识点四：标签“Java”的含义 标签“Java”明确指出了...
使用WebMagic框架实现动态页面爬取技巧
# 1. 动态页面爬取技巧概述 动态页面的爬取对于网络数据采集来说是一个不小的挑战，传统的静态页面爬取技术已经无法满足动态...典型的动态页面有使用 AJAX 技术请求数据呈现，或者通过前端框架如 Angular、Vue、Reac
httpclient代码
05-17
该文件是httpclient的源代码，java类型的
JAVA发送HttpClient请求及接收请求完整代码实例
08-13
NULL 博文链接：https://bijian1013.iteye.com/blog/2310211
httpclient 4.5的源码
08-01
httpclient 4.5的源码，广泛用于java类应用发送http请求，值得学习
httpclient使用实例
HF6787966的博客
10-16 653
导读：本文向我们介绍了如何用httpclient发送Http请求，并举例获取我们项目所在git以及sonar上的数据。   内容： Http协议的重要性相信不用我多说了，HttpClient相比传统JDK自带的URLConnection，增加了易用性和灵活性，它不仅是客户端发送Http请求变得容易，而且也方便了开发人员测试接口（基于Http协议的），即提高了开发的效率，也方便提高代码的健壮性
使用 HttpClient 获取网页 原代码
jq1223的博客
05-20 371
<--httpclient依赖--> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> <version>4.5.12</version> </dependency> 测试： @Test void Test() { .
HttpClient源码分析
MaPengHao的专栏
11-01 1193
本文概述 透过源码分析HttpClient的原理。 使用回顾 public static void sendRequestWithHttpClient() { List<NameValuePair> formparams = new ArrayList<NameValuePair>(); formparams.add(new BasicNameValuePair("account", "")); formparams.add(n..
深入学习 HttpClient 5：源码分析、封装工具类和注意点
BASK2312的博客
05-19 2695
随着现代应用程序对网络通信需求的增加，以及对更灵活、高性能和可定制的 HTTP 客户端的需求，HttpClient 5 成为了 Java 开发者们的首选工具。它提供了强大的功能和灵活的配置选项，使得发送 HTTP 请求和处理响应变得更加简单和高效。在本篇技术博客中，我们将深入学习 HttpClient 5，并从以下几个方面入手：源码分析、封装工具类和最佳实践与注意点。
【java学习】HTTP接口调用：Httpclient、Okhttp、HttpURLConnection、RestTemplate 和 Feign
午后阳光
08-26 5555
HttpClient 是 Apache Jakarta Common 下的子项目，用来提供高效的、最新的、功能丰富的支持 Http 协议的客户端编程工具包，并且它支持 HTTP 协议最新版本和建议。HttpClient 相比传统 JDK 自带的 URLConnection，提升了易用性和灵活性，使客户端发送 HTTP 请求变得容易，提高了开发的效率。首先，对于服务提供者，不需要进行任何配置，feign的访问对于服务提供者其实就是一次http访问。代表一次请求的范围，一般用于请求转发的多个资源中共享数据。
idea如何开启代理
最新发布
win9413的博客
05-07 355
定位到配置菜单后，填写配置。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

黑暗work

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值