Linux内核OOM机制的详细分析

Linux内核OOM机制的详细分析和防止进程被OOM杀死的方法 Linux 内核有个机制叫OOM killer(Out-Of-Memory killer),该机制会监控那些占用内存过大,尤其是瞬间很快消耗大量内存的进程,为了防止内存耗尽而内核会把该进程杀掉。典型的情况是:某天一台机器突然ssh远程登录不了,但能ping通,说明不是网络的故障,原因是sshd进程被OOM killer杀掉了(多次遇到这样的假死状况)。重启机器后查看系统日志/var/log/mes... 阅读详情

Linux内核根据应用程序的要求分配内存,通常来说应用程序分配了内存但是并没有实际全部使用,为了提高性能,这部分没用的内存可以留作它用,这部分内存是属于每个进程的,内核直接回收利用的话比较麻烦,所以内核采用一种过度分配内存(over-commit memory)的办法来间接利用这部分“空闲”的内存,提高整体内存的使用效率。一般来说这样做没有问题,但当大多数应用程序都消耗完自己的内存的时候麻烦就来了,因为这些应用程序的内存需求加起来超出了物理内存(包括swap)的容量,内核(OOM killer)必须杀掉一些进程才能腾出空间保障系统正常运行。用银行的例子来讲可能更容易懂一些,部分人取钱的时候银行不怕,银行有足够的存款应付,当全国人民(或者绝大多数)都取钱而且每个人都想把自己钱取完的时候银行的麻烦就来了,银行实际上是没有这么多钱给大家取的。

比如某天一台机器突然ssh远程登录不了,但能ping通,说明不是网络的故障,原因是sshd进程被OOM killer杀掉了。重启机器后查看系统日志/var/log/messages会发现Out of Memory:Killprocess 1865(sshd)类似的错误信息。又比如有时VPS的MySQL总是无缘无故挂掉,或者VPS 经常死机,登陆到终端发现都是常见的 Out of memory 问题。这通常是因为某时刻应用程序大量请求内存导致系统内存不足造成的,这时会触发 Linux 内核里的 Out of Memory (OOM) killer,OOM killer 会杀掉某个进程以腾出内存留给系统用,不致于让系统立刻崩溃。如果检查相关的日志文件(/var/log/messages)就会看到下面类似的Out of memory:Kill process 信息:

...

Out of memory: Kill process 9682(mysqld) score 9 or sacrifice child

Killed process 9682, UID 27,(mysqld) total-vm:47388kB, anon-rss:3744kB, file-rss:80kB

httpd invoked oom-killer:gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0

httpd cpuset=/ mems_allowed=0

Pid: 8911, comm: httpd Not tainted2.6.32-279.1.1.el6.i686 #1

...

21556 total pagecache pages

21049 pages in swap cache

Swap cache stats: add 12819103,delete 12798054, find 3188096/4634617

Free swap = 0kB

Total swap = 524280kB

131071 pages RAM

0 pages HighMem

3673 pages reserved

67960 pages shared

124940 pages non-shared

Linux内核有个机制叫OOM killer(Out-Of-Memory killer),该机制会监控那些占用内存过大,尤其是瞬间很快消耗大量内存的进程,为了防止内存耗尽内核会把该进程杀掉。

内核检测到系统内存不足、挑选并杀掉某个进程的过程可以参考内核源代码 linux/mm/oom_kill.c,当系统内存不足的时候,out_of_memory()被触发,然后调用 select_bad_process() 选择一个“bad”进程杀掉,判断和选择一个“bad”进程的过程由 oom_badness()决定,最 bad 的那

从零搭建uniapp+vue3+ts+tailwindcss多端兼容开发环境 本文详细介绍了如何从零搭建uniapp+vue3+ts+tailwindcss多端兼容开发环境,包括环境准备、TailwindCSS集成、小程序兼容性解决方案以及样式优化技巧。通过实战步骤和常见问题解答,帮助开发者快速掌握这套技术栈,提升开发效率。 阅读详情

相关推荐

Linux系统下Materials Studio 2020安装避坑指南:从依赖包到许可证配置全流程

本文提供了一份在Linux系统上安装Materials Studio 2020的深度避坑指南。详细解析了从系统依赖包检查、安装流程、许可证配置到Gateway服务部署的全过程,并分享了与Slurm作业调度系统集成的实战经验,旨在帮助用户高效、稳定地完成部署。

weixin_29313547的博客 387

OFDM接收机同步——分组检测

上一篇讲了OFDM发射机中的长短训练序列生成原理,基于此可以得到长短训练序列的准确数值,为接收机中的符号同步中的相关运算提供基础。 进入符号同步前,首先来看OFDM接收机中的第一步——分组检测。分组检测就是检测突发传输方式的信道上是否有新的数据到达,算法的好坏直接影响着接收机的后续工作性能。 一、本文意义 基于《基于xilinx fpga 的ofdm的通信系统基带设计》这本教材中分组检测算法原理,通过matlab编程仿真验证“延时相关+保持长度”分组检测算法的有效性。同时,通过仿真证明教材上的判决公式的问题

boayel的博客 2202

COCO2017数据集类别分布深度解析

本文深入解析了COCO2017数据集的类别分布情况,包括80个常见物体的分类统计和样本数量分析。通过对比训练集与验证集的分布一致性,探讨了数据均衡性问题,并针对高频和低频类别提出了实用的数据预处理和模型训练建议,帮助优化计算机视觉任务的性能。

weixin_42533751的博客 370

爬虫风控实战:当你的代理IP被数美滑块盯上时,如何优雅破解?

本文深入探讨了爬虫风控对抗中数美滑块验证码的破解与防御策略。通过分析数美滑块的多层防御机制,提出了代理IP池精细化管理、请求特征拟人化改造以及工程化滑块验证解决方案等系统性方法,帮助开发者有效降低验证码触发率,提升爬虫效率。

weixin_27419879的博客 207

数据标注的下一代SOP,可能不是Word,而是一段Prompt

再给几个反例:“这个只是正常纹理,不算。真正用于数据生产的Prompt,可能至少包含:任务定义、正例、反例、边界案例、优先级、输出格式、验收逻辑。一个项目100个人,如果100个人对这句话有100种理解,最后返工的就不是一个框,而可能是几十万帧数据。一个项目启动以后,项目经理先研究SOP,培训组长,组长再培训标注员,最后几百个人按照同一套规则干活。实际上一个成熟的项目经理,最重要的能力之一,就是:把客户的业务语言翻译成生产规则。真正让项目难交付的,从来不是80%的普通数据,而是那20%的长尾数据。

AI数据标注猿 297

Codex 100个真实案例 - 用AI自动生成API测试用例(接口测试全覆盖)

API 测试是保障后端质量的关键环节,但手写测试用例既枯燥又容易遗漏。本案例将展示如何用 Codex 从 Swagger/OpenAPI 规范文档自动生成 pytest 测试用例——包括正常流程测试、边界值测试、异常参数测试、性能压力测试,还能自动生成 Mock 数据和精美的 HTML 测试报告。全程自然语言驱动,让 API 测试覆盖率从 0 飙到 90%+!

yueyeguzhuo的专栏 274

从一只龙虾到一支团队:OpenClaw 单 Bot 多 Agent 配置实践

单 Bot 多 Agent只有一个对外的聊天入口(飞书 Bot)内部有多个独立的 AI Agent,各司其职主 Agent(总指挥)负责接收任务并分配给子 Agent子 Agent 完成任务后将结果返回给主 Agent角色Agent ID职责使用场景🎯 总指挥main-agent统筹协调,任务分发接收所有用户请求✍️ 笔杆子内容创作公众号、博客、小红书🔬 参谋canmou深度研究竞品分析、市场调研📋 运营官operations运营管理邮件、日历、社交媒体。

Maynor的博客 1084

Redhawk: 为什么STA CHECK中switch cell覆盖率为0(STA file中switch control pin没有timing window)?

如题,做dynamic analysis时,不需要switch cell的timing window信息,但是做low power analysis需要做上电时switch cell的timing window就是必须的了,本文对switch cell control pin没有timing window (STA CHECK覆盖率为零)的这种现象做分析。

m0_61544122的博客 1197

lwip-2.0.3

lwip是瑞典计算机科学院(SICS)的Adam Dunkels 开发的一个小型开源的TCP/IP协议栈。实现的重点是在保持TCP协议主要功能的基础上减少对RAM 的占用。

jre9下载 jdk9

(Java SE Runtime Environment 9.0) 9.0u175 官方最新版 【x86|x64】

温度PID控制PLC程序

温度PID稳定控制程序,适用于欧姆龙PLC,对其他PLC也具有很大参考价值

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

上一篇: 让服务器底层性能飞起,异步,不一样的感觉
下一篇: 一个很重要但很多人不理解的技术,P2P丨网络穿透与NAT原理分析
Linux服务器开发
博客等级 码龄7年 610粉丝 589原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值