php malloc,PHP7 malloc task buff failed

最新推荐文章于 2022-11-16 15:38:16 发布
最新推荐文章于 2022-11-16 15:38:16 发布
阅读量366 收藏 1
点赞数
文章标签： php malloc
在使用FastDFS 5.0.8版本，搭配PHP7和4GB内存的环境中进行文件上传压力测试时，发现存在内存泄露问题。错误日志显示在`storage_service.c`的第1828行，提示`malloc task buff failed`。PHP代码中尝试通过fastdfs库上传文件并获取信息，但返回了错误信息`Broken pipe`。通过`fdfs_monitor`监控，可以看到两个存储节点的详细状态，包括上传、下载次数和存储空间等信息。问题可能与文件上传过程中的资源管理有关。

摘要生成于 C知道 ，由 DeepSeek-R1 满血版支持， 前往体验 >

benchmark环境：

fastdfs 5.0.8 + PHP7 +4G 内存+ 每次1M文件

用ab做压测的时候，多压测之后发现有内存泄露

ab -c100 -n1000 "http://192.168.10.244/ycf_fastdfs/test.php?fname=001"

日志输出：

···

tail -f /storage/logs/storaged.log

[2016-01-07 18:48:35] ERROR - file: storage_service.c, line: 1828, malloc task buff failed

[2016-01-07 18:48:35] ERROR - file: storage_service.c, line: 1828, malloc task buff failed

[2016-01-07 18:48:35] ERROR - file: storage_service.c, line: 1828, malloc task buff failed

[2016-01-07 18:48:35] ERROR - file: storage_service.c, line: 1828, malloc task buff failed

[2016-01-07 18:48:35] ERROR - file: storage_service.c, line: 1828, malloc task buff failed

[2016-01-07 18:48:35] ERROR - file: storage_service.c, line: 1828, malloc task buff failed

[2016-01-07 18:48:39] ERROR - file: storage_service.c, line: 1828, malloc task buff failed

[2016-01-07 18:48:40] ERROR - file: storage_service.c, line: 1828, malloc task buff failed

[2016-01-07 18:48:54] ERROR - file: storage_service.c, line: 1828, malloc task buff failed

[2016-01-07 18:49:38] ERROR - file: storage_service.c, line: 1828, malloc task buff failed

···

代码

···

$info = $this->fdfs->storage_upload_by_filename($localfile, $ext_name,

array(), null, $this->tracker, $this->storage);

if (is_array($info)) {

$group_name = $info['group_name'];

$remote_filename = $info['filename'];

$source_info = $this->fdfs->get_file_info($group_name,

$remote_filename);

$source_ip = $source_info['source_ip_addr'];

$file_size = $source_info['file_size'];

$this->fdfs->tracker_close_all_connections();

return compact('group_name', 'remote_filename',

'source_ip', 'file_size');

}

···

PHP输出

array('retCode'=>$this->fdfs->get_last_error_no(),'errMsg'=>$this->fdfs->get_last_error_info());

···

array(2) { ["retCode"]=> int(32) ["errMsg"]=> string(11) "Broken pipe" }

···

monitor

···

/usr/bin/fdfs_monitor /etc/fdfs/client.conf

[2016-01-07 18:57:28] DEBUG - base_path=/storage, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

server_count=1, server_index=0

tracker server is 192.168.10.244:22122

group count: 1

Group 1:

group name = group1

disk total space = 465098 MB

disk free space = 291003 MB

trunk free space = 0 MB

storage server count = 2

active server count = 2

storage server port = 23000

storage HTTP port = 8888

store path count = 1

subdir count per path = 256

current write server index = 0

current trunk file id = 0

Storage 1:

id = 192.168.10.242

ip_addr = 192.168.10.242 ACTIVE

http domain =

version = 5.08

join time = 2015-12-29 21:24:45

up time = 2016-01-04 12:18:18

total storage = 934543 MB

free storage = 736624 MB

upload priority = 10

store_path_count = 1

subdir_count_per_path = 256

storage_port = 23000

storage_http_port = 8888

current_write_path = 0

source storage id = 192.168.10.243

if_trunk_server = 0

connection.alloc_count = 256

connection.current_count = 256

connection.max_count = 256

total_upload_count = 401413

success_upload_count = 401413

total_append_count = 0

success_append_count = 0

total_modify_count = 0

success_modify_count = 0

total_truncate_count = 0

success_truncate_count = 0

total_set_meta_count = 286222

success_set_meta_count = 286222

total_delete_count = 16

success_delete_count = 16

total_download_count = 49627

success_download_count = 49627

total_get_meta_count = 21

success_get_meta_count = 21

total_create_link_count = 0

success_create_link_count = 0

total_delete_link_count = 0

success_delete_link_count = 0

total_upload_bytes = 98746663528

success_upload_bytes = 98746663528

total_append_bytes = 0

success_append_bytes = 0

total_modify_bytes = 0

success_modify_bytes = 0

stotal_download_bytes = 2750713415

success_download_bytes = 2750713415

total_sync_in_bytes = 50608869108

success_sync_in_bytes = 50608869108

total_sync_out_bytes = 0

success_sync_out_bytes = 0

total_file_open_count = 1130799

success_file_open_count = 1130799

total_file_read_count = 49648

success_file_read_count = 49648

total_file_write_count = 1589464

success_file_write_count = 1589464

last_heart_beat_time = 2016-01-08 02:58:31

last_source_update = 2016-01-07 18:57:13

last_sync_update = 2016-01-07 18:57:13

last_synced_timestamp = 2016-01-07 18:55:26 (02m:01s delay)

Storage 2:

id = 192.168.10.243

ip_addr = 192.168.10.243 ACTIVE

http domain =

version = 5.08

join time = 2015-12-29 21:09:51

up time = 2016-01-04 20:21:36

total storage = 465098 MB

free storage = 291003 MB

upload priority = 10

store_path_count = 1

subdir_count_per_path = 256

storage_port = 23000

storage_http_port = 8888

current_write_path = 0

source storage id =

if_trunk_server = 0

connection.alloc_count = 256

connection.current_count = 256

connection.max_count = 256

total_upload_count = 403204

success_upload_count = 403204

total_append_count = 0

success_append_count = 0

total_modify_count = 0

success_modify_count = 0

total_truncate_count = 0

success_truncate_count = 0

total_set_meta_count = 282311

success_set_meta_count = 282311

total_delete_count = 0

success_delete_count = 0

total_download_count = 45234

success_download_count = 45234

total_get_meta_count = 0

success_get_meta_count = 0

total_create_link_count = 0

success_create_link_count = 0

total_delete_link_count = 0

success_delete_link_count = 0

total_upload_bytes = 50603269563

success_upload_bytes = 50603269563

total_append_bytes = 0

success_append_bytes = 0

total_modify_bytes = 0

success_modify_bytes = 0

stotal_download_bytes = 2508863994

success_download_bytes = 2508863994

total_sync_in_bytes = 98749240732

success_sync_in_bytes = 98749240732

total_sync_out_bytes = 0

success_sync_out_bytes = 0

total_file_open_count = 1130942

success_file_open_count = 1130942

total_file_read_count = 45234

success_file_read_count = 45234

total_file_write_count = 1593312

success_file_write_count = 1593312

last_heart_beat_time = 2016-01-08 02:58:39

last_source_update = 2016-01-07 18:57:27

last_sync_update = 2016-01-07 18:57:27

last_synced_timestamp = 2016-01-07 18:55:22 (01m:51s delay)

···

制造原理
关注
malloc函数分配内存失败的原因及解决方法
lighthear的博客
04-12 6万+
本文简单描述了malloc()函数分配内存失败的原因以及解决方法，对指针越界造成的分配内存失败做了详细探讨，对之后的内存管理及指针使用很有帮助。
窥探Linux内核进程信息task_struct
singshinesong的博客
02-03 992
突然发觉一直在和Linux死磕，虽然是断断续续的，不过可以称得上是坚持不懈了，甚至有点相爱相杀的感觉。在形影相吊的日子里，作为大龄假程序猿，能够静得下心来玩味Linux内核，失落、慰藉、迷茫、坚持······，个中滋味，一言难尽。也许那本关于Windows驱动的书取名《寒江独钓》，作者也有此意吧：如果告诫读者，如入此门，须当问心无悔！于2022世界杯决战正酣之际，作为连伪球迷都算不上的——假球迷，居然在决赛前由衷地为梅西祈祷夺冠，唯持之以恒、百折不挠、舍身取义的精神所触动而已，无它，当然也包括传说中的精湛球
Fastdfs 关于file: shared_func.c, line: 994, malloc 1 bytes fail问题的解决
冯毅
02-22 5797
今日由于服务器到期没有及时续上，导致服务器被关，服务器启动后发现图片存储出现问题，查看进程，发现无tracker进程，重启无效，查看日志，出现已经信息： [2016-02-22 14:44:07] ERROR - file: shared_func.c, line: 994, malloc 1 bytes fail [2016-02-22 14:44:07] ERROR - file: tra
PHP中内存管理基础知识malloc
casey的博客
09-17 213
malloc内存的管理 void*ptr =malloc(size); free(ptr); PHP7内存接口分配 void*ptr=_emalloc(size); _efree(ptr); 内存预分配：使用mmap分配chunk 内存分类 1.Small（30种规格）（size<=3KB） 2.Larrge(3KB<size<=2MB-4KB) 3.Huge(size>2MB-4KB) ...
memory allocator php,内存泄漏与php扩展中的malloc
weixin_30284125的博客
03-12 202
我做了一个php扩展,看起来像：PHP_FUNCTION(function_name){...proc_data = ( char * )malloc(length);...RETURN_STRINGL( proc_data, length, 1 );}function_name是php代码中可用的函数,它返回一个字符串.但是这个字符串是使用malloc分配内存的,它会自动释放还是我要对它做些什么...
#include <stdio.h> #include <string.h> #include <fcntl.h> #include <errno.h> #include <memory.h> #include <pthread.h> #include <mqueue.h> #define MSGLENGTH 5 #define MSG "hello" static void* send_task(void* argc) { printf("in send_task\n"); mqd_t mqd0, mqd1; struct mq_attr attr; size_t msg_len = 0; int msg_prio = 1; void *recv_buff = NULL; /* 打开消息队列，并获取其属性，分配读缓冲区 */ mqd0 = mq_open("/mqueue_demo0", O_CREAT|O_WRONLY, 0, NULL); mqd1 = mq_open("/mqueue_demo1", O_CREAT|O_RDONLY, 0, NULL); mq_getattr(mqd1, &attr); recv_buff = malloc(attr.mq_msgsize); /* 发送消息 */ mq_send(mqd0, MSG, MSGLENGTH, msg_prio); /* 接收消息 并打印 */ msg_len = mq_receive(mqd1, recv_buff, attr.mq_msgsize, &msg_prio); printf("send_task: msg[%s] msg_length[%d] msg_prio[%d]\n", (char*)recv_buff, msg_len, msg_prio); return NULL; } static void* receive_task(void* argc) { printf("in receive_task\n"); mqd_t mqd0, mqd1; struct mq_attr attr; void *buff = NULL; char msg[10] = {0}; ssize_t msg_len = 0; int msg_prio = 0; /* 打开消息队列，并获取其属性，分配读缓冲区 */ mqd0 = mq_open("/mqueue_demo0", O_RDONLY); mq_getattr(mqd0, &attr); buff = malloc(attr.mq_msgsize); mqd1 = mq_open("/mqueue_demo1", O_WRONLY); /* 接收消息 */ msg_len = mq_receive(mqd0, buff, attr.mq_msgsize, &msg_prio); printf("receive_task: msg[%s] msg_length[%d] msg_prio[%d]\n", (char*)buff, msg_len, msg_prio); /* 验证消息 */ if(strcmp(buff, MSG)==0) { strcpy(msg, "OK"); } else { strcpy(msg, "ERROR"); } /* 发送响应消息 */ mq_send(mqd1, msg, 10, 2); return NULL; } int mqueue_demo(void) { pthread_t tid_send, tid_receive; /* 创建消费者任务 */ pthread_create(&tid_send, NULL, send_task, NULL); pthread_create(&tid_receive, NULL, receive_task, NULL); pthread_join(tid_send, NULL); pthread_join(tid_receive, NULL); return 0; }
最新发布
03-22
例如，send_task中使用malloc分配了recv_buff，根据mq_attr的mq_msgsize属性，这应该没问题。而receive_task中的msg数组大小为10，但发送时使用的大小是10，需要注意是否溢出。 还有线程同步的问题，两个线程可能...
linux top VIRT RES SHR SWAP DATA内存参数详解
热门推荐
鸥鹭忘机
08-09 4万+
Linux top VIRT RES SHR SWAP DATA内存参数详解 其实很早之前就想开博客，写一写码农几年自己积攒下来的知识与见解。看过很多文章有过很多感触，有些收获很值得梳理一下认真思考反思的。今天就先从基本的top开始吧，在网上看了很多关于top讲解内存参数的文章，千篇一律，大部分都是不经深思的转载，自我验证的却只有少数，也许大家忙于工作，急于找到问题并快速解决，而忽略了事后思考总
fastdfs常见故障
CleverCode的博客
12-09 3756
1 机器故障（集群） 1.1 如果集群中，假设其他组的剩余空间都是1%，只剩下一组能够提供服务（一组中有3台机器），如果这组服务中有一台机器挂了，唯一的一个集群是否还能继续服务？ 1.2 如果有2个集群可用，如果其中一个1个集群不可用，是否能够自动切换到另外一个集群？ 2 磁盘故障 2.1 更换磁盘，不能写入问题 3 配置文件不正确 3.1 新加机器，更换磁盘，认为改动配置文件不正确？ 3.2 r...
``` static int s_iServerSocket = 0; static stNetAddr stClient = {0}; //static pthread_t s_stServerThread; int TCPSvr_Run(void) { int iRet = -1; int iClientSocket; int iTaskID = 0; int iCLientNumber = 0; struct sockaddr_in stClientaddr; socklen_t addrlen = sizeof(stClientaddr); if (s_iServerSocket <= 0) { cmn_PrtError("server socket not available"); } if(listen(s_iServerSocket,cmnDfn_MAX_CLIENT) < 0) { cmn_PrtError("listen error"); } while (g_iExit == 0) { if(((iClientSocket = accept(s_iServerSocket, (struct sockaddr *)&stClientaddr,&addrlen))) < 0) { cmn_PrtError("accept failed"); } stClient.iServerSocket = iClientSocket; stClient.iClientNumber = iCLientNumber++; if(ThreadPoolSubmit(TCPRcvTrd, &stClient,&iTaskID) < 0) { cmn_PrtError("Error in submitting task to thread pool"); } if(ThreadPoolSubmit(TCPSndTrd, &stClient,&iTaskID) < 0) { cmn_PrtError("Error in submitting task to thread pool"); } } iRet = 0; _Exit: return iRet; } static int s_iClientSocket; static struct sockaddr_in stServerAddr; static stNetAddr stServer = {0}; //static pthread_t s_iClientPhread; int TCPClnt_Run(void) { int iRet = -1; int iTaskID = 0; //struct sockaddr_in stServerAddr; if(s_iClientSocket == -1) { cmn_PrtError("Client socket not available"); } if(connect(s_iClientSocket, (struct sockaddr *)&stServerAddr, sizeof(stServerAddr)) < 0) { cmn_PrtError("Error in connecting to server"); } stServer.iServerSocket = s_iClientSocket; if(ThreadPoolSubmit(TCPSndTrd,&stServer,&iTaskID) < 0) { cmn_PrtError("Error in submitting task to thread pool"); } if (ThreadPoolSubmit(TCPRcvTrd,&stServer,&iTaskID) < 0) { cmn_PrtError("Error in submitting task to thread pool"); } while (g_iExit == 0) { } iRet = 0; _Exit: return iRet; } void* TCPRcvTrd(void *arg) { //int iSockfd = *(int*)arg; //int iSockfd = (int)(intptr_t)arg; //int iLoopNum = 0; stNetAddr * ServerAddr = (stNetAddr *)arg; int iSockfd = ServerAddr->iServerSocket; int iClientNum = ServerAddr->iClientNumber; char crcvBuffer[cmnDfn_BUFF_LEN]; int iBytesReceived; unsigned long long ulTotalBytes = 0; struct timeval stStartTime; struct timeval stEndTime; //struct timeval stDisplayedTime = {0}; float fElapsedTime; float fNetSpeed; gettimeofday(&stStartTime, NULL); while (g_iExit == 0) { if((iBytesReceived = recv(iSockfd, crcvBuffer, sizeof(crcvBuffer), 0)) < 0) { cmn_PrtError("Error in receiving data"); } else if(iBytesReceived == 0) { printf("Client disconnected\n"); break; } ulTotalBytes += iBytesReceived; gettimeofday(&stEndTime,NULL); fElapsedTime = (stEndTime.tv_sec - stStartTime.tv_sec) + (stEndTime.tv_usec - stStartTime.tv_usec) / 1000000.0; fNetSpeed = ((float)ulTotalBytes * 8) / (fElapsedTime * 1000 * 1000); if (fElapsedTime >= DISPLAY_INTERVAL) { //printf("Received: %d bytes\n", ulTotalBytes); if(ServerAddr->iByteNumber != 0) { printf("Client %d Net Speed: %.2f Mbps\r", iClientNum,fNetSpeed); fflush(stdout); } else if(ServerAddr->iByteNumber == 0) { printf("Net Speed: %.2f Mbps\n", fNetSpeed); fflush(stdout); } stStartTime = stEndTime; ulTotalBytes = 0; } } _Exit: //close(iSockfd); pthread_exit(NULL); }```检查代码是否有bug并给出解决方案
03-21
cmn_PrtError("accept failed"); close(s_iServerSocket); // 关闭服务端 socket goto _Exit; } ``` --- ### **四、潜在的内存泄漏（中危错误）** **问题定位**： 动态分配的 `stNetAddr` 结构体在线程结束后未...
FastDFS之Storage程序框架
hfty290的专栏
12-20 3181
FastDFS分布式文件系统中有Tracker和Storage两个角色，其中绝大多数功能都是在Storage中实现，包括网络处理、文件上传、下载、同步、磁盘恢复等众多功能，本篇将讲述Storage程序的启动过程，网络处理，磁盘处理，与Tracker通讯，文件同步等过程。
那一定都是你的错！- 一次FastDFS并发问题的排查经历
AFSGEFEGH的博客
10-12 1754
那一定都是你的错！- 一次FastDFS并发问题的排查经历 前一段时间，业务部门同事反馈在一次生产服务器升级之后，POS消费上传小票业务偶现异常，上传小票业务有重试机制，有些重试三次也不会成功，他们排查了一下没有找到原因，希望架构部帮忙解决。 公司使用的是FastDFS来做的图片服务器，生产使用了六台服务器外加一个存储，集群采用的是：2个tracker+4个storage，storage分为两个group，使用独立的nginx做文件代理访问。各软件版本信息如下： 操作系统：centos6.9 FastDFS
mysql 最大连接数max_connections解决办法
a15804226768的博客
11-16 1万+
1.问题原因1）客户端程序在退出之前没有调用mysql_close()。[写程序的疏忽，或者数据库的db类库没有自动关闭每次的连接]2)客户端sleep的时间在wait_timeout或interactive_timeout规定的秒内没有发出任何请求到服务器。[类似常连，类似于不完整的tcp ip协议构造，服务端一直认为客户端仍然存在(有可能客户端已经断掉了)]3)客户端程序在结束之前向服务器发送了请求还没得到返回结果就结束掉了。[参看：tcp ip协议的三次握手]2.解决办法。
fasdfs源码分析
西京刀客
10-26 1040
代码目录结构 参考 fastdfs-nginx扩展模块源码分析 参考URL: https://www.cnblogs.com/littleatp/p/4361318.html
FastDFS分布式文件系统问题总汇
avj64803的博客
06-18 373
定位问题首先要看日志文件。出现问题时，先检查返回的错误号和错误信息。然后查看服务器端日志，相信可以定位到问题所在。 1. FastDFS适用的场景以及不适用的场景？ FastDFS是为互联网应用量身定做的一套分布式文件存储系统，非常适合用来存储用户图片、视频、文档等文件。对于互联网应用，和其他分布式文件系统相比，优势非常明显。具体情况大家可以看相关的介绍文档，包括FastDFS介...
FDFS部署中遇到的几个bug坑，已解决
思研
03-29 1万+
1.不能上传文件到storage：,tracker_query_storage fail, error no: 2, error info: No such file or directory 问题： 1.storage的配置文件中图片的保存路径中的文件夹名字写错（注意：这个问题是最容易出现的） 2. storage的防火墙没有关闭 解决： 1. 关闭防火墙 2.查看配置文件，错误的地...
FastDFS安装教程
我的IT人生
01-13 7255
工作环境 操作系统： CentOS 6.7 64bit编译工具： git, C/C++ 编译工具集(gcc/make等等)libfastcommon： V1.24FastDFS： V5.0.7fastdfs-nginx-moduleNginx 1.8.1注明: 本教程三台linux服务器,安装两台storage服务,安装一台tracker服务.(测试在storage服务器上面测试) 安装
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值