Maximum limits (in macOS file descriptors)

Operating systems (Linux and macOS included) have settings which limit the number of files and processes that are allowed to be open. This limit protects the system from being overrun. But its default is usually set too low, when machines had way less power. Thus a “gotcha” that is only apparent when “too many files open” crashes appear only under load (as in during a stress test or production spike).

man bash, the Bash manual says the ulimit command (common among Linux flavors) provides “control over the resources available to the shell and to processes started by it”.

  1. Obtain the current limit of file descriptors

    ulimit -n

    An example output: “256” or “10032”.

    PROTIP: On MacOS, the maximum number that can be specified is 12288.

  2. Obtain the current limit of processes

    ulimit -u

    An example output: “1418”.

On macOS

  1. Obtain the current limit:

    launchctl limit maxfiles

    The response output should have numbers like this:

    maxfiles    65536          200000

    The first number is the “soft” limit and the second number is the “hard” limit.

    Configuration changes are necessary if lower numbers are displayed, such as:

    maxfiles    256            unlimited
  2. If the soft limit is too low (such as 256), set the current session to:

    sudo launchctl limit maxfiles 65536 200000

    Some set it to 1048576 (over a million).

    Since sudo is needed, you are prompted for a password.

    PROTIP: Because this would go back to defaults on reboot, add this command in your ~/.bash_profile

    Alternately, install Facebook’s Watchman utility which watches and adjusts automatically.

    PROTIP: Take both a full/complete backup to ensure fall-back. Also run a benchmark performance measurement before and after changing the configuration to detect issues.

  3. Over several versions, Apple has been changing the way system-wide open file limits can be set upon restart.

    Yosemite and older

    NOTE: On older MacOSX Yosemite, to make the settings permanent, increase the limits, edit (or create) file /etc/launchd.conf to contain:

    limit maxfiles 65536 200000

    Sierra and newer versions

    Newer versions of macOS do not reference the file due to security considerations.

    On newer macOS Sierra and High Sierra, Dejan Kitic and this found that two plist files need to be added.

  4. Copy in folder /Library/LaunchDaemons/ plist files from a GitHub repository:

    https://github.com/wilsonmar/mac-setup/blob/master/configs/limit.maxfiles.plist at 524288

    https://github.com/wilsonmar/mac-setup/blob/master/configs/limit.maxproc.plist at 2048

  5. Invoke the files:

    sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist
    sudo launchctl load -w /Library/LaunchDaemons/limit.maxproc.plist

    The files’ ownership needs to be changed to “root:wheel”.

    Their permissions need to be “-rw-r–r–”, set by sudo chmod 644.

  6. So how do you turn csrutil off? Google says “sudo csrutil disable…”. But not so easy. That can only be done in Recovery Mode. So, reboot, hold command + R to enter Recovery Mode, once there open terminal and do csrutil disable… Finally a breakthrough…disabled.

  7. Now you can adjust the process limit on Mac OS X Yosemite and El Capitan versions:

    sudo ulimit -n 65536 200000

    Since sudo is needed, you are prompted for a password.

  8. To increase the inotify watchers max limit, edit (or create) file /etc/sysctl.conf (inherited from BSD) to contain:

    kern.maxfiles=49152
    kern.maxfilesperproc=24576

    or

    kern.maxfiles=200000
    kern.maxfilesperproc=200000

    Alternately, run interactive commands:

    sudo sysctl -w kern.maxfiles=5242880

    The response:

    kern.maxfiles: 49152 -> 5242880
    sudo sysctl -w kern.maxfilesperproc=524288

    The response:

    kern.maxfilesperproc: 24576 -> 524288
  9. Restart the system for the new limits to take effect.

  10. After restarting, verify the new limits by running:

On RedHat and CentOS

  1. To enable PAM-based user limits so that non-root users, such as the riak user, may specify a higher value for maximum open files, edit (or create) file /etc/security/limits.conf to contain:

    * soft nofile 65536
    * hard nofile 200000

    If the file already exists, append the above to the bottom of the file.

  2. To activate limits, edit (or create) file /etc/pam.d/login to contain:

    session    required   pam_limits.so
  3. Restart the machine.
  4. Verify by “ulimit -a” again.

    ulimit -n
    ulimit -Hn # Hard limit
    ulimit -Sn # Soft limit

    If the response is “10032”

macOS 解决 too many open files

转:macOS 解决 too many open files | 比十万多一点

报错 too many open files 大致有以下三种可能 [1] [2] [3]
\1. 操作系统打开的文件句柄数过多(内核的限制)
\2. launchd 对进程进行了限制
\3. shell 对进程进行了限制

内核的限制

整个操作系统可以打开的文件数受内核参数影响,可以通过以下命令查看

$ sysctl kern.maxfiles
$ sysctl kern.maxfilesperproc

在我电脑上输出如下

kern.maxfiles: 49152
kern.maxfilesperproc: 42576

好像已经很大了。

如果需要临时修改的话,运行如下命令

sudo sysctl -w kern.maxfiles=20480 # 或其他你选择的数字
sudo sysctl -w kern.maxfilesperproc=18000 # 或其他你选择的数字

永久修改,需要在 /etc/sysctl.conf 里加上类似的下述内容

kern.maxfiles=20480
kern.maxfilesperproc=18000

这个文件可能需要自行创建

launchd 对进程的限制

获取当前的限制:

launchctl limit maxfiles

输出类似这样:

maxfiles    256            unlimited

其中前一个是软限制,后一个是硬件限制。

临时修改:

sudo launchctl limit maxfiles 65536 200000

系统范围内修改则需要在文件夹 /Library/LaunchDaemons 下创建一个 plist 文件 limit.maxfiles.plist

<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>limit.maxfiles</string>
    <key>ProgramArguments</key>
    <array>
      <string>launchctl</string>
      <string>limit</string>
      <string>maxfiles</string>
      <string>65536</string>
      <string>200000</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>ServiceIPC</key>
    <false/>
  </dict>
</plist>

修改文件权限

sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist
sudo chmod 644 /Library/LaunchDaemons/limit.maxfiles.plist

载入新设定

sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist

shell 的限制

通过下述命令查看现有限制

ulimit -a

得到如下输出

-n: file descriptors                256

通过 ulimit -S -n 4096 来修改。如果需要保持修改,可以将这一句命令加入你的 .bash_profile 或 .zshrc 等。

总结

一般来说,修改了上述三个限制,重启一下,这个问题就可以解决了。

在实际操作中,我修改到第二步重启,第三个就自动修改了。

macos - 为什么使用 'launchctl limit' 将 maxfiles 的硬限制设置为 “无限制” 会导致硬限制略高于软限制?- 询问不同

评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值