【GreatSQL优化器-09】make_join_query_block

【GreatSQL优化器-09】make_join_query_block

一、make_join_query_block介绍

GreatSQL优化器对于多张表join的连接顺序在前面的章节介绍过的best_access_path函数已经执行了,接着就是把where条件进行切割然后推给合适的表。这个过程就是由函数make_join_query_block来执行的。

下面用几个简单的例子来说明join连接中条件推送是什么。

CREATE TABLE t1 (c1 INT PRIMARY KEY, c2 INT,date1 DATETIME);
INSERT INTO t1 VALUES (1,10,'2021-03-25 16:44:00.123456'),(2,1,'2022-03-26 16:44:00.123456'),(3,4,'2023-03-27 16:44:00.123456'),(5,5,'2024-03-25 16:44:00.123456'),(7,null,'2020-03-25 16:44:00.123456'),(8,10,'2020-10-25 16:44:00.123456'),(11,16,'2023-03-25 16:44:00.123456');
CREATE TABLE t2 (cc1 INT PRIMARY KEY, cc2 INT);
INSERT INTO t2 VALUES (1,3),(2,1),(3,2),(4,3),(5,15);
CREATE TABLE t3 (ccc1 INT, ccc2 varchar(100));
INSERT INTO t3 VALUES (1,'aa1'),(2,'bb1'),(3,'cc1'),(4,'dd1'),(null,'ee');
CREATE INDEX idx1 ON t1(c2);
CREATE INDEX idx2 ON t1(c2,date1);
CREATE INDEX idx2_1 ON t2(cc2);
CREATE INDEX idx3_1 ON t3(ccc1);

下面这个例子((t1.c1 = t3.ccc1) or (t3.ccc1 < 3))条件推送给t1

greatsql> EXPLAIN FORMAT=TREE SELECT * FROM t1 join t3 ON t1.c1=t3.ccc1 or t3.ccc1<3;
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| EXPLAIN                                                                                                                                                                                                                                                         |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| -> Filter: ((t1.c1 = t3.ccc1) or (t3.ccc1 < 3))  (cost=5.26 rows=35)
    -> Inner hash join (no condition)  (cost=5.26 rows=35)
        -> Index scan on t1 using idx2  (cost=0.34 rows=7)
        -> Hash
            -> Table scan on t3  (cost=0.75 rows=5)
 |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

下面例子(t1.c1 < 3)条件推给t1,(ccc1=t1.c1)条件推给t3

greatsql> EXPLAIN FORMAT=TREE SELECT * FROM t1 join t3 ON t1.c1=t3.ccc1 and t3.ccc1<3;
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| EXPLAIN                                                                                                                                                                                                                          |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| -> Nested loop inner join  (cost=2.40 rows=2)
    -> Filter: (t1.c1 < 3)  (cost=1.70 rows=2)
        -> Index scan on t1 using idx2  (cost=1.70 rows=7)
    -> Index lookup on t3 using idx3_1 (ccc1=t1.c1)  (cost=0.30 rows=1)
 |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

下面例子((t3.ccc1 = t1.c2) or (t3.ccc1 is null) or (t3.ccc2 like 'a%'))条件推给t3,(((t3.ccc1 = t1.c2) and (t2.

# Other default tuning values # MySQL Server Instance Configuration File # ---------------------------------------------------------------------- # Generated by the MySQL Server Instance Configuration Wizard # # # Installation Instructions # ---------------------------------------------------------------------- # # On Linux you can copy this file to /etc/my.cnf to set global options, # mysql-data-dir/my.cnf to set server-specific options # (@localstatedir@ for this installation) or to # ~/.my.cnf to set user-specific options. # # On Windows, when MySQL has been installed using MySQL Installer you # should keep this file in the ProgramData directory of your server # (e.g. C:\ProgramData\MySQL\MySQL Server X.Y). To make sure the server # reads the config file, use the startup option "--defaults-file". # # To run the server from the command line, execute this in a # command line shell, e.g. # mysqld --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini" # # To install the server as a Windows service manually, execute this in a # command line shell, e.g. # mysqld --install MySQLXY --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini" # # And then execute this in a command line shell to start the server, e.g. # net start MySQLXY # # # Guidelines for editing this file # ---------------------------------------------------------------------- # # In this file, you can use all long options that the program supports. # If you want to know the options a program supports, start the program # with the "--help" option. # # More detailed information about the individual options can also be # found in the manual. # # For advice on how to change settings please see # https://dev.mysql.com/doc/refman/8.0/en/server-configuration-defaults.html # # # CLIENT SECTION # ---------------------------------------------------------------------- # # The following options will be read by MySQL client applications. # Note that only client applications shipped by MySQL are guaranteed # to read this section. If you want your own MySQL client program to # honor these values, you need to specify it as an option during the # MySQL client library initialization. # [client] # pipe= # socket=MYSQL port=3306 [mysql] no-beep # default-character-set= # SERVER SECTION # ---------------------------------------------------------------------- # # The following options will be read by the MySQL Server. Make sure that # you have installed the server correctly (see above) so it reads this # file. # [mysqld] port = 3306 bind-address = 0.0.0.0 # The next three options are mutually exclusive to SERVER_PORT below. # skip-networking # enable-named-pipe # shared-memory # shared-memory-base-name=MYSQL # The Pipe the MySQL Server will use. # socket=MYSQL # The access control granted to clients on the named pipe created by the MySQL Server. # named-pipe-full-access-group= # The TCP/IP Port the MySQL Server will listen on port=3306 # Path to installation directory. All paths are usually resolved relative to this. # basedir="D:/mysql" # Path to the database root datadir=D:/mysql\Data # The default character set that will be used when a new schema or table is # created and no character set is defined # character-set-server= # The default storage engine that will be used when create new tables when default-storage-engine=INNODB # The current server SQL mode, which can be set dynamically. # Modes affect the SQL syntax MySQL supports and the data validation checks it performs. This # makes it easier to use MySQL in different environments and to use MySQL together with other # database servers. sql-mode="ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION" # General and Slow logging. log-output=FILE general-log=0 general_log_file="WIN-20240617SLP.log" slow-query-log=1 slow_query_log_file="WIN-20240617SLP-slow.log" long_query_time=10 # Error Logging. log-error="WIN-20240617SLP.err" # ***** Group Replication Related ***** # Specifies the base name to use for binary log files. With binary logging # enabled, the server logs all statements that change data to the binary # log, which is used for backup and replication. log-bin="WIN-20240617SLP-bin" # ***** Group Replication Related ***** # Specifies the server ID. For servers that are used in a replication topology, # you must specify a unique server ID for each replication server, in the # range from 1 to 2^32 − 1. "Unique" means that each ID must be different # from every other ID in use by any other source or replica. server-id=1 # Indicates how table and database names are stored on disk and used in MySQL. # Value 0 = Table and database names are stored on disk using the lettercase specified in the CREATE # TABLE or CREATE DATABASE statement. Name comparisons are case-sensitive. You should not # set this variable to 0 if you are running MySQL on a system that has case-insensitive file # names (such as Windows or macOS). If you force this variable to 0 with # --lower-case-table-names=0 on a case-insensitive file system and access MyISAM tablenames # using different lettercases, index corruption may result. # Value 1 = Table names are stored in lowercase on disk and name comparisons are not case-sensitive. # MySQL converts all table names to lowercase on storage and lookup. This behavior also applies # to database names and table aliases. # Value 2 = Table and database names are stored on disk using the lettercase specified in the CREATE TABLE # or CREATE DATABASE statement, but MySQL converts them to lowercase on lookup. Name comparisons # are not case-sensitive. This works only on file systems that are not case-sensitive! InnoDB # table names and view names are stored in lowercase, as for lower_case_table_names=1. lower_case_table_names=1 # This variable is used to limit the effect of data import and export operations, such as # those performed by the LOAD DATA and SELECT ... INTO OUTFILE statements and the # LOAD_FILE() function. These operations are permitted only to users who have the FILE privilege. secure-file-priv="D:/mysql/Uploads" # The maximum amount of concurrent sessions the MySQL server will # allow. One of these connections will be reserved for a user with # SUPER privileges to allow the administrator to login even if the # connection limit has been reached. max_connections=151 # The number of open tables for all threads. Increasing this value increases the number # of file descriptors that mysqld requires. table_open_cache=4000 # Defines the maximum amount of memory that can be occupied by the TempTable # storage engine before it starts storing data on disk. temptable_max_ram=1G # Defines the maximum size of internal in-memory temporary tables created # by the MEMORY storage engine and, as of MySQL 8.0.28, the TempTable storage # engine. If an internal in-memory temporary table exceeds this size, it is # automatically converted to an on-disk internal temporary table. tmp_table_size=128M # The storage engine for in-memory internal temporary tables (see Section 8.4.4, "Internal # Temporary Table Use in MySQL"). Permitted values are TempTable (the default) and MEMORY. internal_tmp_mem_storage_engine=TempTable #*** MyISAM Specific options # The maximum size of the temporary file that MySQL is permitted to use while re-creating a # MyISAM index (during REPAIR TABLE, ALTER TABLE, or LOAD DATA). If the file size would be # larger than this value, the index is created using the key cache instead, which is slower. # The value is given in bytes. myisam_max_sort_file_size=2146435072 # The size of the buffer that is allocated when sorting MyISAM indexes during a REPAIR TABLE # or when creating indexes with CREATE INDEX or ALTER TABLE. myisam_sort_buffer_size=245M # Size of the Key Buffer, used to cache index blocks for MyISAM tables. # Do not set it larger than 30% of your available memory, as some memory # is also required by the OS to cache rows. Even if you're not using # MyISAM tables, you should still set it to 8-64M as it will also be # used for internal temporary disk tables. key_buffer_size=8M # Each thread that does a sequential scan for a MyISAM table allocates a buffer # of this size (in bytes) for each table it scans. If you do many sequential # scans, you might want to increase this value, which defaults to 131072. The # value of this variable should be a multiple of 4KB. If it is set to a value # that is not a multiple of 4KB, its value is rounded down to the nearest multiple # of 4KB. read_buffer_size=128K # This variable is used for reads from MyISAM tables, and, for any storage engine, # for Multi-Range Read optimization. read_rnd_buffer_size=256K #*** INNODB Specific options *** # innodb_data_home_dir= # Use this option if you have a MySQL server with InnoDB support enabled # but you do not plan to use it. This will save memory and disk space # and speed up some things. # skip-innodb # If set to 1, InnoDB will flush (fsync) the transaction logs to the # disk at each commit, which offers full ACID behavior. If you are # willing to compromise this safety, and you are running small # transactions, you may set this to 0 or 2 to reduce disk I/O to the # logs. Value 0 means that the log is only written to the log file and # the log file flushed to disk approximately once per second. Value 2 # means the log is written to the log file at each commit, but the log # file is only flushed to disk approximately once per second. innodb_flush_log_at_trx_commit=1 # The size in bytes of the buffer that InnoDB uses to write to the log files on # disk. The default value changed from 8MB to 16MB with the introduction of 32KB # and 64KB innodb_page_size values. A large log buffer enables large transactions # to run without the need to write the log to disk before the transactions commit. # Thus, if you have transactions that update, insert, or delete many rows, making # the log buffer larger saves disk I/O. innodb_log_buffer_size=16M # The size in bytes of the buffer pool, the memory area where InnoDB caches table # and index data. The default value is 134217728 bytes (128MB). The maximum value # depends on the CPU architecture; the maximum is 4294967295 (232-1) on 32-bit systems # and 18446744073709551615 (264-1) on 64-bit systems. On 32-bit systems, the CPU # architecture and operating system may impose a lower practical maximum size than the # stated maximum. When the size of the buffer pool is greater than 1GB, setting # innodb_buffer_pool_instances to a value greater than 1 can improve the scalability on # a busy server. innodb_buffer_pool_size=128M # Defines the amount of disk space occupied by redo log files. This variable supersedes the # innodb_log_files_in_group and innodb_log_file_size variables. innodb_redo_log_capacity=100M # Defines the maximum number of threads permitted inside of InnoDB. A value # of 0 (the default) is interpreted as infinite concurrency (no limit). This # variable is intended for performance tuning on high concurrency systems. # InnoDB tries to keep the number of threads inside InnoDB less than or equal to # the innodb_thread_concurrency limit. Once the limit is reached, additional threads # are placed into a "First In, First Out" (FIFO) queue for waiting threads. Threads # waiting for locks are not counted in the number of concurrently executing threads. innodb_thread_concurrency=25 # The increment size (in MB) for extending the size of an auto-extend InnoDB system tablespace file when it becomes full. innodb_autoextend_increment=64 # The number of regions that the InnoDB buffer pool is divided into. # For systems with buffer pools in the multi-gigabyte range, dividing the buffer pool into separate instances can improve concurrency, # by reducing contention as different threads read and write to cached pages. innodb_buffer_pool_instances=8 # Determines the number of threads that can enter InnoDB concurrently. innodb_concurrency_tickets=5000 # Specifies how long in milliseconds (ms) a block inserted into the old sublist must stay there after its first access before # it can be moved to the new sublist. innodb_old_blocks_time=1000 # When this variable is enabled, InnoDB updates statistics during metadata statements. innodb_stats_on_metadata=0 # When innodb_file_per_table is enabled (the default in 5.6.6 and higher), InnoDB stores the data and indexes for each newly created table # in a separate .ibd file, rather than in the system tablespace. innodb_file_per_table=1 # Use the following list of values: 0 for crc32, 1 for strict_crc32, 2 for innodb, 3 for strict_innodb, 4 for none, 5 for strict_none. innodb_checksum_algorithm=0 # If this is set to a nonzero value, all tables are closed every flush_time seconds to free up resources and # synchronize unflushed data to disk. # This option is best used only on systems with minimal resources. flush_time=0 # The minimum size of the buffer that is used for plain index scans, range index scans, and joins that do not use # indexes and thus perform full table scans. join_buffer_size=256K # The maximum size of one packet or any generated or intermediate string, or any parameter sent by the # mysql_stmt_send_long_data() C API function. max_allowed_packet=64M # If more than this many successive connection requests from a host are interrupted without a successful connection, # the server blocks that host from performing further connections. max_connect_errors=100 # The number of file descriptors available to mysqld from the operating system # Try increasing the value of this option if mysqld gives the error "Too many open files". open_files_limit=8161 # If you see many sort_merge_passes per second in SHOW GLOBAL STATUS output, you can consider increasing the # sort_buffer_size value to speed up ORDER BY or GROUP BY operations that cannot be improved with query optimization # or improved indexing. sort_buffer_size=256K # Specify the maximum size of a row-based binary log event, in bytes. # Rows are grouped into events smaller than this size if possible. The value should be a multiple of 256. binlog_row_event_max_size=8K # If the value of this variable is greater than 0, a replica synchronizes its master.info file to disk. # (using fdatasync()) after every sync_source_info events. sync_source_info=10000 # If the value of this variable is greater than 0, the MySQL server synchronizes its relay log to disk. # (using fdatasync()) after every sync_relay_log writes to the relay log. sync_relay_log=10000 # Load mysql plugins at start."plugin_x ; plugin_y". # plugin_load # The TCP/IP Port the MySQL Server X Protocol will listen on. 这就是配置文件内容
10-03
### 配置文件解读 #### 客户端部分 ```ini [client] port=3306 [mysql] no-beep ``` - `port=3306`:指定客户端连接 MySQL 服务器时使用的端口号为 3306。 - `no-beep`:禁止 MySQL 客户端在出现错误时发出蜂鸣声。 #### 服务器部分 ```ini [mysqld] port = 3306 bind-address = 0.0.0.0 ``` - `port = 3306`:MySQL 服务器监听的 TCP/IP 端口号为 3306。 - `bind-address = 0.0.0.0`:表示服务器监听所有可用的网络接口,允许任何 IP 地址的客户端连接。 ```ini datadir=D:/mysql\Data ``` - `datadir`:指定 MySQL 数据库文件的存储路径为 `D:/mysql\Data`。 ```ini default-storage-engine=INNODB ``` - `default-storage-engine`:设置创建新表时默认使用的存储引擎为 InnoDB。 ```ini sql-mode="ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION" ``` - `sql-mode`:定义了 MySQL 服务器的 SQL 模式,影响 SQL 语法和数据验证检查。例如 `ONLY_FULL_GROUP_BY` 要求 `GROUP BY` 语句中的非聚合列必须出现在 `GROUP BY` 子句中。 ```ini log-output=FILE general-log=0 general_log_file="WIN-20240617SLP.log" slow-query-log=1 slow_query_log_file="WIN-20240617SLP-slow.log" long_query_time=10 ``` - `log-output=FILE`:指定日志输出到文件。 - `general-log=0`:关闭通用查询日志。 - `general_log_file`:通用查询日志的文件名。 - `slow-query-log=1`:开启慢查询日志。 - `slow_query_log_file`:慢查询日志的文件名。 - `long_query_time=10`:定义查询执行时间超过 10 秒的查询为慢查询。 ```ini log-error="WIN-20240617SLP.err" ``` - `log-error`:指定错误日志的文件名。 ```ini log-bin="WIN-20240617SLP-bin" ``` - `log-bin`:指定二进制日志的文件名,用于备份和复制。 ```ini server-id=1 ``` - `server-id`:指定服务器的唯一 ID,用于复制拓扑,范围从 1 到 2^32 - 1。 ```ini lower_case_table_names=1 ``` - `lower_case_table_names`:设置表名和数据库名在磁盘上的存储方式和比较方式。值为 1 表示表名存储为小写,比较时不区分大小写。 ```ini secure-file-priv="D:/mysql/Uploads" ``` - `secure-file-priv`:限制 `LOAD DATA` 和 `SELECT ... INTO OUTFILE` 等操作可以访问的文件目录为 `D:/mysql/Uploads`,增强安全性[^2]。 ```ini max_connections=151 ``` - `max_connections`:设置 MySQL 服务器允许的最大并发连接数为 151。 #### MyISAM 特定选项 ```ini myisam_max_sort_file_size=2146435072 myisam_sort_buffer_size=245M key_buffer_size=8M read_buffer_size=128K read_rnd_buffer_size=256K ``` - `myisam_max_sort_file_size`:指定在重新创建 MyISAM 索引时允许使用的临时文件的最大大小。 - `myisam_sort_buffer_size`:在排序 MyISAM 索引时分配的缓冲区大小。 - `key_buffer_size`:用于缓存 MyISAM 表索引块的键缓冲区大小。 - `read_buffer_size`:顺序扫描 MyISAM 表时每个线程分配的缓冲区大小。 - `read_rnd_buffer_size`:用于读取 MyISAM 表和多范围读取优化的缓冲区大小。 #### InnoDB 特定选项 ```ini innodb_flush_log_at_trx_commit=1 innodb_log_buffer_size=16M innodb_buffer_pool_size=128M innodb_redo_log_capacity=100M innodb_thread_concurrency=25 innodb_autoextend_increment=64 innodb_buffer_pool_instances=8 innodb_concurrency_tickets=5000 innodb_old_blocks_time=1000 innodb_stats_on_metadata=0 innodb_file_per_table=1 innodb_checksum_algorithm=0 ``` - `innodb_flush_log_at_trx_commit`:控制事务提交时日志文件的刷新方式,值为 1 表示每次提交都将事务日志刷新到磁盘,保证 ACID 特性。 - `innodb_log_buffer_size`:InnoDB 用于写入日志文件的缓冲区大小。 - `innodb_buffer_pool_size`:InnoDB 缓存表和索引数据的缓冲区池大小。 - `innodb_redo_log_capacity`:重做日志文件占用的磁盘空间大小。 - `innodb_thread_concurrency`:限制进入 InnoDB 的最大线程数。 - `innodb_autoextend_increment`:InnoDB 系统表空间文件自动扩展时的增量大小。 - `innodb_buffer_pool_instances`:将 InnoDB 缓冲区池划分为的区域数量,提高并发性能。 - `innodb_concurrency_tickets`:控制可以同时进入 InnoDB 的线程数。 - `innodb_old_blocks_time`:指定块插入旧子列表后首次访问到可以移动到新子列表的时间间隔。 - `innodb_stats_on_metadata`:控制是否在元数据语句期间更新统计信息。 - `innodb_file_per_table`:开启后,InnoDB 为每个新创建的表使用单独的 `.ibd` 文件存储数据和索引。 - `innodb_checksum_algorithm`:指定 InnoDB 的校验和算法。 ### 优化建议 #### 内存相关优化 - **innodb_buffer_pool_size**:如果服务器内存充足,可以适当增大该值,例如设置为物理内存的 70% - 80%,以提高 InnoDB 表的缓存命中率。 ```ini innodb_buffer_pool_size=2G ``` - **key_buffer_size**:如果使用 MyISAM 表较多,可以适当增大该值,但不要超过可用内存的 30%。 ```ini key_buffer_size=64M ``` #### 并发相关优化 - **max_connections**:根据实际业务需求和服务器性能,调整最大并发连接数。如果并发连接数经常达到上限,可以适当增大该值。 ```ini max_connections=500 ``` - **innodb_thread_concurrency**:根据服务器的 CPU 核心数和并发情况,调整该值。一般可以设置为 CPU 核心数的 2 - 4 倍。 ```ini innodb_thread_concurrency=48 ``` #### 日志相关优化 - **innodb_flush_log_at_trx_commit**:如果对数据安全性要求不是特别高,可以将该值设置为 2 或 0,以减少磁盘 I/O。 ```ini innodb_flush_log_at_trx_commit=2 ``` #### 慢查询日志优化 - **long_query_time**:可以适当降低该值,例如设置为 1 秒,以便更及时地发现慢查询。 ```ini long_query_time=1 ```
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值