(转) Linux Increase The Maximum Number Of Open Files

调整Linux最大打开文件数
本文介绍如何在Linux系统中调整单个用户可打开的最大文件数。通过修改系统配置文件和使用ulimit命令,可以有效地增加允许同时打开的文件数量,以满足高负载应用程序的需求。
Issue:
How do I set the maximum number of files allowed to be open on a system
Resolution:
The current setting for maximum number of open files can be viewed with the command:

ulimit -n

This number indicates the maximum number of files normal users (i.e. non-root) can have open in a single session. Note that for the root user, ulimit -n will sometimes output 1024 even after following the procedure to increase the maximum number of open files. This won't effect root's ability to open large numbers of files, as only normal users are bound by this value.

To increase the maximum number of open files beyond the default of 1024, two changes to the system may be necessary. In these examples, we will increase the maximum number of open files to the arbitrary value of 2048. All changes need to be made by the root user and users will need to log out and log back in before the changes will take effect.

1. Configure the system to accept the desired value for maximum number of open files Check the value in /proc/sys/fs/file-max to see if it is larger than the value needed for the maximum number of open files:

# cat /proc/sys/fs/file-max

If the value isn't large enough, echo an appropriate number into the variable and add the change to /etc/sysctl.conf to make it persistent across reboots. If the number is already larger than the value you wish to use, skip to step 2.

# echo 2048 > /proc/sys/fs/file-max

and edit /etc/sysctl.conf to include the line:

fs.file-max = 2048

2. Set the value for maximum number of open files In the file /etc/security/limits.conf, below the commented line that reads

#<domain> <type> <item> <value>

add this line:

* - nofile 2048

This line sets the default number of open file descriptors for every user on the system to 2048. Note that the "nofile" item has two possible limit values under the <type> header: hard and soft. Both types of limits must be set before the change in the maximum number of open files will take effect. By using the "-" character, both hard and soft limits are set simultaneously.

The hard limit represents the maximum value a soft limit may have and the soft limit represents the limit being actively enforced on the system at that time. Hard limits can be lowered by normal users, but not raised and soft limits cannot be set higher than hard limits. Only root may raise hard limits.

When increasing file limit descriptors, you may want to simply double the value. For example, if you need to increase the default value of 1024, increase the value to 2048 first. If you need to increase it again, try 4096, etc.


==================================================================
1。ulimit -a 查看系统目前资源限制的设定。
# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 7168
virtual memory (kbytes, -v) unlimited
#
通过以上命令,我们可以看到open files 的最大数为1024
那么我们可以通过一下命令修改该参数的最大值
2. ulimit -n 4096
# ulimit -n 4096
# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 7168
virtual memory (kbytes, -v) unlimited

这样我们就修改了系统在同一时间打开文件资源的最大数,基本解决以上问题。

另外我们可以通过lsof -p [进程的 pid]来查看系统当前打开的文件资源,可以了解不同时期系统的文件资源的使用情况,可根据情况进行系统资源的配置。
===================================================================
从这里学到的linux的命令
查看进程所占的文件描述符 lsof -p 390 | wc -l
查看系统的各种资源限制或者更改这个限制 ulimit -a
# Other default tuning values # MySQL Server Instance Configuration File # ---------------------------------------------------------------------- # Generated by the MySQL Server Instance Configuration Wizard # # # Installation Instructions # ---------------------------------------------------------------------- # # On Linux you can copy this file to /etc/my.cnf to set global options, # mysql-data-dir/my.cnf to set server-specific options # (@localstatedir@ for this installation) or to # ~/.my.cnf to set user-specific options. # # On Windows, when MySQL has been installed using MySQL Installer you # should keep this file in the ProgramData directory of your server # (e.g. C:\ProgramData\MySQL\MySQL Server X.Y). To make sure the server # reads the config file, use the startup option "--defaults-file". # # To run the server from the command line, execute this in a # command line shell, e.g. # mysqld --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini" # # To install the server as a Windows service manually, execute this in a # command line shell, e.g. # mysqld --install MySQLXY --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini" # # And then execute this in a command line shell to start the server, e.g. # net start MySQLXY # # # Guidelines for editing this file # ---------------------------------------------------------------------- # # In this file, you can use all long options that the program supports. # If you want to know the options a program supports, start the program # with the "--help" option. # # More detailed information about the individual options can also be # found in the manual. # # For advice on how to change settings please see # https://dev.mysql.com/doc/refman/8.0/en/server-configuration-defaults.html # # # CLIENT SECTION # ---------------------------------------------------------------------- # # The following options will be read by MySQL client applications. # Note that only client applications shipped by MySQL are guaranteed # to read this section. If you want your own MySQL client program to # honor these values, you need to specify it as an option during the # MySQL client library initialization. # [client] # pipe= # socket=MYSQL port=3306 [mysql] no-beep # default-character-set= # SERVER SECTION # ---------------------------------------------------------------------- # # The following options will be read by the MySQL Server. Make sure that # you have installed the server correctly (see above) so it reads this # file. # [mysqld] port = 3306 bind-address = 0.0.0.0 # The next three options are mutually exclusive to SERVER_PORT below. # skip-networking # enable-named-pipe # shared-memory # shared-memory-base-name=MYSQL # The Pipe the MySQL Server will use. # socket=MYSQL # The access control granted to clients on the named pipe created by the MySQL Server. # named-pipe-full-access-group= # The TCP/IP Port the MySQL Server will listen on port=3306 # Path to installation directory. All paths are usually resolved relative to this. # basedir="D:/mysql" # Path to the database root datadir=D:/mysql\Data # The default character set that will be used when a new schema or table is # created and no character set is defined # character-set-server= # The default storage engine that will be used when create new tables when default-storage-engine=INNODB # The current server SQL mode, which can be set dynamically. # Modes affect the SQL syntax MySQL supports and the data validation checks it performs. This # makes it easier to use MySQL in different environments and to use MySQL together with other # database servers. sql-mode="ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION" # General and Slow logging. log-output=FILE general-log=0 general_log_file="WIN-20240617SLP.log" slow-query-log=1 slow_query_log_file="WIN-20240617SLP-slow.log" long_query_time=10 # Error Logging. log-error="WIN-20240617SLP.err" # ***** Group Replication Related ***** # Specifies the base name to use for binary log files. With binary logging # enabled, the server logs all statements that change data to the binary # log, which is used for backup and replication. log-bin="WIN-20240617SLP-bin" # ***** Group Replication Related ***** # Specifies the server ID. For servers that are used in a replication topology, # you must specify a unique server ID for each replication server, in the # range from 1 to 2^32 − 1. "Unique" means that each ID must be different # from every other ID in use by any other source or replica. server-id=1 # Indicates how table and database names are stored on disk and used in MySQL. # Value 0 = Table and database names are stored on disk using the lettercase specified in the CREATE # TABLE or CREATE DATABASE statement. Name comparisons are case-sensitive. You should not # set this variable to 0 if you are running MySQL on a system that has case-insensitive file # names (such as Windows or macOS). If you force this variable to 0 with # --lower-case-table-names=0 on a case-insensitive file system and access MyISAM tablenames # using different lettercases, index corruption may result. # Value 1 = Table names are stored in lowercase on disk and name comparisons are not case-sensitive. # MySQL converts all table names to lowercase on storage and lookup. This behavior also applies # to database names and table aliases. # Value 2 = Table and database names are stored on disk using the lettercase specified in the CREATE TABLE # or CREATE DATABASE statement, but MySQL converts them to lowercase on lookup. Name comparisons # are not case-sensitive. This works only on file systems that are not case-sensitive! InnoDB # table names and view names are stored in lowercase, as for lower_case_table_names=1. lower_case_table_names=1 # This variable is used to limit the effect of data import and export operations, such as # those performed by the LOAD DATA and SELECT ... INTO OUTFILE statements and the # LOAD_FILE() function. These operations are permitted only to users who have the FILE privilege. secure-file-priv="D:/mysql/Uploads" # The maximum amount of concurrent sessions the MySQL server will # allow. One of these connections will be reserved for a user with # SUPER privileges to allow the administrator to login even if the # connection limit has been reached. max_connections=151 # The number of open tables for all threads. Increasing this value increases the number # of file descriptors that mysqld requires. table_open_cache=4000 # Defines the maximum amount of memory that can be occupied by the TempTable # storage engine before it starts storing data on disk. temptable_max_ram=1G # Defines the maximum size of internal in-memory temporary tables created # by the MEMORY storage engine and, as of MySQL 8.0.28, the TempTable storage # engine. If an internal in-memory temporary table exceeds this size, it is # automatically converted to an on-disk internal temporary table. tmp_table_size=128M # The storage engine for in-memory internal temporary tables (see Section 8.4.4, "Internal # Temporary Table Use in MySQL"). Permitted values are TempTable (the default) and MEMORY. internal_tmp_mem_storage_engine=TempTable #*** MyISAM Specific options # The maximum size of the temporary file that MySQL is permitted to use while re-creating a # MyISAM index (during REPAIR TABLE, ALTER TABLE, or LOAD DATA). If the file size would be # larger than this value, the index is created using the key cache instead, which is slower. # The value is given in bytes. myisam_max_sort_file_size=2146435072 # The size of the buffer that is allocated when sorting MyISAM indexes during a REPAIR TABLE # or when creating indexes with CREATE INDEX or ALTER TABLE. myisam_sort_buffer_size=245M # Size of the Key Buffer, used to cache index blocks for MyISAM tables. # Do not set it larger than 30% of your available memory, as some memory # is also required by the OS to cache rows. Even if you're not using # MyISAM tables, you should still set it to 8-64M as it will also be # used for internal temporary disk tables. key_buffer_size=8M # Each thread that does a sequential scan for a MyISAM table allocates a buffer # of this size (in bytes) for each table it scans. If you do many sequential # scans, you might want to increase this value, which defaults to 131072. The # value of this variable should be a multiple of 4KB. If it is set to a value # that is not a multiple of 4KB, its value is rounded down to the nearest multiple # of 4KB. read_buffer_size=128K # This variable is used for reads from MyISAM tables, and, for any storage engine, # for Multi-Range Read optimization. read_rnd_buffer_size=256K #*** INNODB Specific options *** # innodb_data_home_dir= # Use this option if you have a MySQL server with InnoDB support enabled # but you do not plan to use it. This will save memory and disk space # and speed up some things. # skip-innodb # If set to 1, InnoDB will flush (fsync) the transaction logs to the # disk at each commit, which offers full ACID behavior. If you are # willing to compromise this safety, and you are running small # transactions, you may set this to 0 or 2 to reduce disk I/O to the # logs. Value 0 means that the log is only written to the log file and # the log file flushed to disk approximately once per second. Value 2 # means the log is written to the log file at each commit, but the log # file is only flushed to disk approximately once per second. innodb_flush_log_at_trx_commit=1 # The size in bytes of the buffer that InnoDB uses to write to the log files on # disk. The default value changed from 8MB to 16MB with the introduction of 32KB # and 64KB innodb_page_size values. A large log buffer enables large transactions # to run without the need to write the log to disk before the transactions commit. # Thus, if you have transactions that update, insert, or delete many rows, making # the log buffer larger saves disk I/O. innodb_log_buffer_size=16M # The size in bytes of the buffer pool, the memory area where InnoDB caches table # and index data. The default value is 134217728 bytes (128MB). The maximum value # depends on the CPU architecture; the maximum is 4294967295 (232-1) on 32-bit systems # and 18446744073709551615 (264-1) on 64-bit systems. On 32-bit systems, the CPU # architecture and operating system may impose a lower practical maximum size than the # stated maximum. When the size of the buffer pool is greater than 1GB, setting # innodb_buffer_pool_instances to a value greater than 1 can improve the scalability on # a busy server. innodb_buffer_pool_size=128M # Defines the amount of disk space occupied by redo log files. This variable supersedes the # innodb_log_files_in_group and innodb_log_file_size variables. innodb_redo_log_capacity=100M # Defines the maximum number of threads permitted inside of InnoDB. A value # of 0 (the default) is interpreted as infinite concurrency (no limit). This # variable is intended for performance tuning on high concurrency systems. # InnoDB tries to keep the number of threads inside InnoDB less than or equal to # the innodb_thread_concurrency limit. Once the limit is reached, additional threads # are placed into a "First In, First Out" (FIFO) queue for waiting threads. Threads # waiting for locks are not counted in the number of concurrently executing threads. innodb_thread_concurrency=25 # The increment size (in MB) for extending the size of an auto-extend InnoDB system tablespace file when it becomes full. innodb_autoextend_increment=64 # The number of regions that the InnoDB buffer pool is divided into. # For systems with buffer pools in the multi-gigabyte range, dividing the buffer pool into separate instances can improve concurrency, # by reducing contention as different threads read and write to cached pages. innodb_buffer_pool_instances=8 # Determines the number of threads that can enter InnoDB concurrently. innodb_concurrency_tickets=5000 # Specifies how long in milliseconds (ms) a block inserted into the old sublist must stay there after its first access before # it can be moved to the new sublist. innodb_old_blocks_time=1000 # When this variable is enabled, InnoDB updates statistics during metadata statements. innodb_stats_on_metadata=0 # When innodb_file_per_table is enabled (the default in 5.6.6 and higher), InnoDB stores the data and indexes for each newly created table # in a separate .ibd file, rather than in the system tablespace. innodb_file_per_table=1 # Use the following list of values: 0 for crc32, 1 for strict_crc32, 2 for innodb, 3 for strict_innodb, 4 for none, 5 for strict_none. innodb_checksum_algorithm=0 # If this is set to a nonzero value, all tables are closed every flush_time seconds to free up resources and # synchronize unflushed data to disk. # This option is best used only on systems with minimal resources. flush_time=0 # The minimum size of the buffer that is used for plain index scans, range index scans, and joins that do not use # indexes and thus perform full table scans. join_buffer_size=256K # The maximum size of one packet or any generated or intermediate string, or any parameter sent by the # mysql_stmt_send_long_data() C API function. max_allowed_packet=64M # If more than this many successive connection requests from a host are interrupted without a successful connection, # the server blocks that host from performing further connections. max_connect_errors=100 # The number of file descriptors available to mysqld from the operating system # Try increasing the value of this option if mysqld gives the error "Too many open files". open_files_limit=8161 # If you see many sort_merge_passes per second in SHOW GLOBAL STATUS output, you can consider increasing the # sort_buffer_size value to speed up ORDER BY or GROUP BY operations that cannot be improved with query optimization # or improved indexing. sort_buffer_size=256K # Specify the maximum size of a row-based binary log event, in bytes. # Rows are grouped into events smaller than this size if possible. The value should be a multiple of 256. binlog_row_event_max_size=8K # If the value of this variable is greater than 0, a replica synchronizes its master.info file to disk. # (using fdatasync()) after every sync_source_info events. sync_source_info=10000 # If the value of this variable is greater than 0, the MySQL server synchronizes its relay log to disk. # (using fdatasync()) after every sync_relay_log writes to the relay log. sync_relay_log=10000 # Load mysql plugins at start."plugin_x ; plugin_y". # plugin_load # The TCP/IP Port the MySQL Server X Protocol will listen on. 这就是配置文件内容
最新发布
10-03
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值