Operating systems (Linux and macOS included) have settings which limit the number of files and processes that are allowed to be open. This limit protects the system from being overrun. But its default is usually set too low, when machines had way less power. Thus a “gotcha” that is only apparent when “too many files open” crashes appear only under load (as in during a stress test or production spike).
man bash, the Bash manual says the ulimit command (common among Linux flavors) provides “control over the resources available to the shell and to processes started by it”.
-
Obtain the current limit of file descriptors
ulimit -nAn example output: “256” or “10032”.
PROTIP: On MacOS, the maximum number that can be specified is 12288.
-
Obtain the current limit of processes
ulimit -uAn example output: “1418”.
On macOS
-
Obtain the current limit:
launchctl limit maxfilesThe response output should have numbers like this:
maxfiles 65536 200000The first number is the “soft” limit and the second number is the “hard” limit.
Configuration changes are necessary if lower numbers are displayed, such as:
maxfiles 256 unlimited -
If the soft limit is too low (such as 256), set the current session to:
sudo launchctl limit maxfiles 65536 200000Some set it to 1048576 (over a million).
Since sudo is needed, you are prompted for a password.
PROTIP: Because this would go back to defaults on reboot, add this command in your ~/.bash_profile
Alternately, install Facebook’s Watchman utility which watches and adjusts automatically.
PROTIP: Take both a full/complete backup to ensure fall-back. Also run a benchmark performance measurement before and after changing the configuration to detect issues.
-
Over several versions, Apple has been changing the way system-wide open file limits can be set upon restart.
Yosemite and older
NOTE: On older MacOSX Yosemite, to make the settings permanent, increase the limits, edit (or create) file /etc/launchd.conf to contain:
limit maxfiles 65536 200000Sierra and newer versions
Newer versions of macOS do not reference the file due to security considerations.
On newer macOS Sierra and High Sierra, Dejan Kitic and this found that two plist files need to be added.
-
Copy in folder /Library/LaunchDaemons/ plist files from a GitHub repository:
https://github.com/wilsonmar/mac-setup/blob/master/configs/limit.maxfiles.plist at 524288
https://github.com/wilsonmar/mac-setup/blob/master/configs/limit.maxproc.plist at 2048
-
Invoke the files:
sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist sudo launchctl load -w /Library/LaunchDaemons/limit.maxproc.plistThe files’ ownership needs to be changed to “root:wheel”.
Their permissions need to be “-rw-r–r–”, set by sudo chmod 644.
-
So how do you turn csrutil off? Google says “sudo csrutil disable…”. But not so easy. That can only be done in Recovery Mode. So, reboot, hold command + R to enter Recovery Mode, once there open terminal and do csrutil disable… Finally a breakthrough…disabled.
-
Now you can adjust the process limit on Mac OS X Yosemite and El Capitan versions:
sudo ulimit -n 65536 200000Since sudo is needed, you are prompted for a password.
-
To increase the inotify watchers max limit, edit (or create) file /etc/sysctl.conf (inherited from BSD) to contain:
kern.maxfiles=49152 kern.maxfilesperproc=24576or
kern.maxfiles=200000 kern.maxfilesperproc=200000Alternately, run interactive commands:
sudo sysctl -w kern.maxfiles=5242880The response:
kern.maxfiles: 49152 -> 5242880sudo sysctl -w kern.maxfilesperproc=524288The response:
kern.maxfilesperproc: 24576 -> 524288 -
Restart the system for the new limits to take effect.
-
After restarting, verify the new limits by running:
On RedHat and CentOS
-
To enable PAM-based user limits so that non-root users, such as the riak user, may specify a higher value for maximum open files, edit (or create) file /etc/security/limits.conf to contain:
* soft nofile 65536 * hard nofile 200000If the file already exists, append the above to the bottom of the file.
-
To activate limits, edit (or create) file /etc/pam.d/login to contain:
session required pam_limits.so - Restart the machine.
-
Verify by “ulimit -a” again.
ulimit -n ulimit -Hn # Hard limit ulimit -Sn # Soft limitIf the response is “10032”
macOS 解决 too many open files
转:macOS 解决 too many open files | 比十万多一点
报错 too many open files 大致有以下三种可能 [1] [2] [3]:
\1. 操作系统打开的文件句柄数过多(内核的限制)
\2. launchd 对进程进行了限制
\3. shell 对进程进行了限制
内核的限制
整个操作系统可以打开的文件数受内核参数影响,可以通过以下命令查看
$ sysctl kern.maxfiles $ sysctl kern.maxfilesperproc |
在我电脑上输出如下
kern.maxfiles: 49152 kern.maxfilesperproc: 42576 |
好像已经很大了。
如果需要临时修改的话,运行如下命令
sudo sysctl -w kern.maxfiles=20480 # 或其他你选择的数字 sudo sysctl -w kern.maxfilesperproc=18000 # 或其他你选择的数字 |
永久修改,需要在 /etc/sysctl.conf 里加上类似的下述内容
kern.maxfiles=20480 kern.maxfilesperproc=18000 |
这个文件可能需要自行创建
launchd 对进程的限制
获取当前的限制:
launchctl limit maxfiles |
输出类似这样:
maxfiles 256 unlimited |
其中前一个是软限制,后一个是硬件限制。
临时修改:
sudo launchctl limit maxfiles 65536 200000 |
系统范围内修改则需要在文件夹 /Library/LaunchDaemons 下创建一个 plist 文件 limit.maxfiles.plist:
<plist version="1.0">
<dict>
<key>Label</key>
<string>limit.maxfiles</string>
<key>ProgramArguments</key>
<array>
<string>launchctl</string>
<string>limit</string>
<string>maxfiles</string>
<string>65536</string>
<string>200000</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>ServiceIPC</key>
<false/>
</dict>
</plist>
|
修改文件权限
sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist sudo chmod 644 /Library/LaunchDaemons/limit.maxfiles.plist |
载入新设定
sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist |
shell 的限制
通过下述命令查看现有限制
ulimit -a |
得到如下输出
-n: file descriptors 256 |
通过 ulimit -S -n 4096 来修改。如果需要保持修改,可以将这一句命令加入你的 .bash_profile 或 .zshrc 等。
总结
一般来说,修改了上述三个限制,重启一下,这个问题就可以解决了。
在实际操作中,我修改到第二步重启,第三个就自动修改了。
macos - 为什么使用 'launchctl limit' 将 maxfiles 的硬限制设置为 “无限制” 会导致硬限制略高于软限制?- 询问不同
3309

被折叠的 条评论
为什么被折叠?



