使用CURL不当，导致的CLOSE_WAIT.

今天，系统日志里显示 Too many open files，于是使用lsof命令查看文件句柄数。发现是网络连接太多（CLOSE_WAIT).

 

linux平台下：使用下面命令

lsof -nl|grep CLOSE_WAIT|awk '{print $9,$10}'|wc -l

发现 CLOSE_WAIT|太多

展示片段如下：

192.168.10.32:52727->192.168.10.31:webcache (CLOSE_WAIT)

192.168.10.32:52747->192.168.10.31:webcache (CLOSE_WAIT)

 

 我这是处理逻辑问题，curl_easy_init 和curl_easy_cleanup没有成对调用，

因触发异常导致的。解决办法是封装一个类（实际上，这个非常类似于java中的try{}finally{}用法，但比JAVA的看起来舒服

class AutoDestory{

public:

//构造函数中加入curl_easy_init

//析构函数加入curl_easy_cleanup
}

但考虑到程序使用CURL进行HTTP连接，ftp连接等是一个常见问题，于是搜索CURL和CLOSE_WAIT相关资料，归纳下。

 

【1】官方bug讨论区：

http://curl.haxx.se/mail/lib-2013-11/0006.html

curl-library Mailing List Archives

 

Many CLOSE_WAIT when handling lots of URLs

[PATCH] Cleanup dead connections in CLOSE_WAIT state sooner

这个是个bug，但是这个说的是"multi"的接口，与”easy“接口无关，且这个出现在 7.29.0.版本，且现在已经修复

 

curl-library Mailing List Archives

 

Re: [ curl-Bugs-1358860 ] libcurl not closing sockets

这个是一个2005年的帖子。。。。

1) libcurl only supervises sockets between a request and its response. 

libcurl reads, writes and "supervises" the sockets during the whole operation: 
the request and the subsequent response. 

If the server keeps the connection open, so will libcurl to be able to re-use 
the connection if the application decides to do so. 

When libcurl has completed a transfer and leaves the connection open, there is 
no "supervision" of any sockets as libcurl has no magic threads or any another 
means of doing that. 

When you then later on call curl_easy_perform() again with a URL that is fit 
for connection re-use it checks if the connection previously left alive is 
still alive and if so it re-uses it. 

> 2) if the server closes a socket because it has been idle for X amount of 
> time, libcurl will not be supervising the socket, hence not detect the 
> close. 

Correct. It won't detect that until you try to re-use that connection again. 

> 3) to make libcurl supervise the socket during this time (i.e. after 
> receiving the request) would be a fundamental change to the way libcurl is 
> currently designed. Perhaps even introducing a performance hit. 

Yes. It would require a separate thread or similar that would detect when the 
peer closes the connection. 

 

这篇说明，CURL没有去侦测CLOSE_WAIT,除非单独需要一个独立线程

【2】sourceforge说明

http://sourceforge.net/p/curl/bugs/558/

#558 CLOSE_WAIT problem

How many threads? How many sockets are in CLOSE_WAIT?

libcurl keep sockets "open" by default after a
curl_easy_perform(), to be prepared to re-use the connection
on a subsequent call to it again. But if the call doesn't
come within a limited time, the remote server might decide
to close down the connection anyway and the socket ends up
in CLOSE_WAIT.

You can avoid this by forcing libcurl to close the sockets
after use, which then of course disables persistent
connections on subsequent calls.

Can you reproduce this problem with a small stand-alone app
that you can provide the source for？

这里说了CLOSE_WAIT产生的原因，curl_easy_perform()默认会暂时延长关闭连接，这是为了下次请求可以重用该连接。但如果client端在限定时间内并没有数据传输，远程服务器会主动关闭该连接，这时连接的SOCKET就变成CLOSE_WAIT状态了。

下面是一个较好的解释：（注，这个也是他/她转载别人的）

hwz_119 写道

CLOSE_WAIT问题解决
http://blog.youkuaiyun.com/hwz119/article/details/1611229

 

http://blog.youkuaiyun.com/hwz119/article/details/1611229

解决办法：

【方法1】

【官方】

http://curl.haxx.se/libcurl/c/CURLOPT_FORBID_REUSE.html

但里面有提醒，这个可能会对性能有很大影响,另外，这个与curl_easy_cleanup函数没有关系。

【方法2】

每次CURL都重新curl_easy_initcurl用完都调用curl_easy_cleanup