记一次Java多线程程序调试经历:HttpClient 死锁

在运行自编写的Java爬虫程序时,遇到了程序卡死的情况,通过jstatd和VisualVM分析发现存在死锁。线程状态显示主线程因等待锁而阻塞,问题根源在于HttpClient。解决方案包括:升级HttpClient库版本、确保及时关闭连接以及设置请求超时,以避免类似死锁问题。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

我自己写了一个爬虫程序,跑了半天后程序就卡死了,没有任何输出和动静。
先是使用jstatdVisualVM,参考这篇文章:jstatd,VisualVM使用和报错解决
结果如下图:
在这里插入图片描述
看到内存毫无变化,应该是发生了死锁。


使用命令jstack查看线程堆栈状态:

-> % jstack 26924      
2019-12-23 09:53:38
Full thread dump OpenJDK 64-Bit Server VM (11.0.4+11-post-Ubuntu-1ubuntu218.04.3 mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x00007f35dc0035b0, length=20, elements={
   
   
0x00007f360c011000, 0x00007f360c0b0800, 0x00007f360c0b2800, 0x00007f360c0b8800,
0x00007f360c0ba800, 0x00007f360c0bd000, 0x00007f360c0bf000, 0x00007f360c0f9000,
0x00007f360c0fe800, 0x00007f360c20f800, 0x00007f360c249000, 0x00007f360c378800,
0x00007f360c38b000, 0x00007f360c389800, 0x00007f360c390000, 0x00007f360c391800,
0x00007f360c393000, 0x00007f360c395800, 0x00007f360c38e000, 0x00007f35dc001800
}

"main" #1 prio=5 os_prio=0 cpu=697257.22ms elapsed=65383.05s tid=0x00007f360c011000 nid=0x692d waiting on condition  [0x00007f3612f97000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.4/Native Method)
	- parking to wait for  <0x00000000e3c00020> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.4/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.4/AbstractQueuedSynchronizer.java:2081)
	at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:379)
	at org.apache.http.pool.AbstractConnPool.access$200(AbstractConnPool.java:69)
	at org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java:245)
	- locked <0x00000000e3c000b0> (a org.apache.http.pool.AbstractConnPool$2)
	at org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java:193)
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:304)
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:280)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
	at com.zhang.spider.getHtml(spider.java:219)
	at com.zhang.spider.main(spider.java:145)


"pool-1-thread-1" #13 prio=5 os_prio=0 cpu=68909.14ms elapsed=65380.72s tid=0x00007f360c378800 nid=0x693f waiting on condition  [0x00007f35f0c38000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.4/Native Method)
	- parking to wait for  <0x00000000ecfae070> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.4/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.4/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take(java.base@11.0.4/LinkedBlockingQueue.java:433)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.4/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.4/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.4/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.4/Thread.java:834)
	
JNI global refs: 17, weak refs: 0

看到线程池中的线程状态是java.lang.Thread.State: WAITING (parking)parking to wait for <0x00000000ecfae070> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject),这是正常的线程等待状态。

发现主线程有问题,虽然同样是java.lang.Thread.State: WAITING (parking)
但在等锁parking to wait for <0x00000000e3c00020> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

发现问题是由org.apache.http.impl.client.CloseableHttpClient造成的。
然后看到了这篇文章:记一次 HttpClient 死锁问题 | Mr.Kail’s Blog


一个解决办法是升级库的版本:

    <dependency>
      <groupId>org.apache.httpcomponents</groupId>
      <artifactId>httpclient</artifactId>
      <version>4.5.10</version>
    </dependency>

另一个办法是记得关闭连接:

try (CloseableHttpClient httpclient = HttpClients.createDefault()) {
   
   
	CloseableHttpResponse response = httpclient.execute(url);
	response.close()
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值