图解MySQL | [原理解析] MySQL组提交(group commit)

本文围绕MySQL展开,在设置crash safe相关参数为双1的前提下,介绍了WAL机制、Redo log和组提交的作用。重点阐述了binlog组提交的三个阶段(Flush、Sync、Commit)及各阶段队列的作用,最后分析了Sync阶段参数导致的bug,该bug已在部分版本修复。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

本文转载自[图解MySQL],原创作者:王悦,黄炎,周海鸣

 

​引  言

本文是由爱可生研发团队出品的「图解MySQL」系列文章,不定期更新,但篇篇精品。

爱可生开源社区持续运营维护的小目标:

  • 每周至少推送一篇高质量技术文章

  • 每月研发团队发布开源组件新版

  • 每年1024开源一款企业级组件

  • 2019年至少25场社区活动

欢迎大家持续关注~


前提:

  • 以下讨论的前提 是设置MySQL的crash safe相关参数为双1:

sync_binlog=1

innodb_flush_log_at_trx_commit=1

 

 

背景说明:

  • WAL机制 (Write Ahead Log)定义:

    WAL指的是对数据文件进行修改前,必须将修改先记录日志。MySQL为了保证ACID中的一致性和持久性,使用了WAL。

  • Redo log的作用:

    Redo log就是一种WAL的应用。当数据库忽然掉电,再重新启动时,MySQL可以通过Redo log还原数据。也就是说,每次事务提交时,不用同步刷新磁盘数据文件,只需要同步刷新Redo log就足够了。相比写数据文件时的随机IO,写Redo log时的顺序IO能够提高事务提交速度。

  • 组提交的作用:

1)在没有开启binlog时

Redo log的刷盘操作将会是最终影响MySQL TPS的瓶颈所在。为了缓解这一问题,MySQL使用了组提交,将多个刷盘操作合并成一个,如果说10个事务依次排队刷盘的时间成本是10,那么将这10个事务一次性一起刷盘的时间成本则近似于1。

2)当开启binlog时

为了保证Redo log和binlog的数据一致性,MySQL使用了二阶段提交,由binlog作为事务的协调者。而 引入二阶段提交 使得binlog又成为了性能瓶颈,先前的Redo log 组提交 也成了摆设。为了再次缓解这一问题,MySQL增加了binlog的组提交,目的同样是将binlog的多个刷盘操作合并成一个,结合Redo log本身已经实现的 组提交,分为三个阶段(Flush 阶段、Sync 阶段、Commit 阶段)完成binlog 组提交,最大化每次刷盘的收益,弱化磁盘瓶颈,提高性能。

 

图解:

下图我们假借“渡口运输”的例子来看看binlog 组提交三个阶段的流程:

在MySQL中每个阶段都有一个队列,每个队列都有一把锁保护,第一个进入队列的事务会成为leader,leader领导所在队列的所有事务,全权负责整队的操作,完成后通知队内其他事务操作结束。

 

Flush 阶段 (图中第一个渡口)

  • 首先获取队列中的事务组

  • 将Redo log中prepare阶段的数据刷盘(图中Flush Redo log)

  • 将binlog数据写入文件,当然此时只是写入文件系统的缓冲,并不能保证数据库崩溃时binlog不丢失 (图中Write binlog)

  • Flush阶段队列的作用是提供了Redo log的组提交

  • 如果在这一步完成后数据库崩溃,由于协调者binlog中不保证有该组事务的记录,所以MySQL可能会在重启后回滚该组事务

 

Sync 阶段 (图中第二个渡口)

  • 这里为了增加一组事务中的事务数量,提高刷盘收益,MySQL使用两个参数控制获取队列事务组的时机:

        binlog_group_commit_sync_delay=N:在等待N μs后,开始事务刷盘(图中Sync binlog)

        binlog_group_commit_sync_no_delay_count=N:如果队列中的事务数达到N个,就忽视binlog_group_commit_sync_delay的设置,直接开始刷盘(图中Sync binlog)

  • Sync阶段队列的作用是支持binlog的组提交

  • 如果在这一步完成后数据库崩溃,由于协调者binlog中已经有了事务记录,MySQL会在重启后通过Flush 阶段中Redo log刷盘的数据继续进行事务的提交

 

Commit 阶段 (图中第三个渡口)

  • 首先获取队列中的事务组

  • 依次将Redo log中已经prepare的事务在引擎层提交(图中InnoDB Commit)

  • Commit阶段不用刷盘,如上所述,Flush阶段中的Redo log刷盘已经足够保证数据库崩溃时的数据安全了

  • Commit阶段队列的作用是承接Sync阶段的事务,完成最后的引擎提交,使得Sync可以尽早的处理下一组事务,最大化组提交的效率

 

缺陷分析:

本文最后要讨论的bug(可通过阅读原文查看)就是来源于Sync 阶段中的那个binlog参数binlog_group_commit_sync_delay,在MySQL 5.7.19中,如果该参数不为10的倍数,则会导致事务在Sync 阶段等待极大的时间,表现出来的现象就是执行的sql长时间无法返回。该bug已在MySQL 5.7.24和8.0.13被修复。

-- 创建数据库 CREATE DATABASE TicketSalesSystem; GO USE TicketSalesSystem; GO -- 创建票价类型表 CREATE TABLE TicketType ( TypeID INT PRIMARY KEY IDENTITY(1,1), TypeName NVARCHAR(20) NOT NULL CHECK (TypeName IN ('老年', '小孩', '成人', '团体')), BasePrice DECIMAL(10,2) NOT NULL, DiscountRate DECIMAL(5,2) DEFAULT 1.0, Description NVARCHAR(255), CreatedAt DATETIME DEFAULT GETDATE(), UpdatedAt DATETIME DEFAULT GETDATE() ); GO -- 创建营业员表 CREATE TABLE Salesperson ( SalespersonID INT PRIMARY KEY IDENTITY(1,1), EmployeeID NVARCHAR(20) UNIQUE NOT NULL, Name NVARCHAR(50) NOT NULL, Password NVARCHAR(100) NOT NULL, Role NVARCHAR(20) DEFAULT 'staff' CHECK (Role IN ('admin', 'staff')), IsActive BIT DEFAULT 1, CreatedAt DATETIME DEFAULT GETDATE(), LastLogin DATETIME NULL ); GO -- 创建门票销售表 CREATE TABLE TicketSale ( SaleID INT PRIMARY KEY IDENTITY(1,1), TypeID INT NOT NULL, SalespersonID INT NOT NULL, Quantity INT NOT NULL CHECK (Quantity > 0), UnitPrice DECIMAL(10,2) NOT NULL, TotalAmount DECIMAL(10,2) NOT NULL, SaleTime DATETIME DEFAULT GETDATE(), IsRefund BIT DEFAULT 0, CONSTRAINT FK_TicketSale_TicketType FOREIGN KEY (TypeID) REFERENCES TicketType(TypeID), CONSTRAINT FK_TicketSale_Salesperson FOREIGN KEY (SalespersonID) REFERENCES Salesperson(SalespersonID) ); GO -- 创建退票记录表 CREATE TABLE RefundRecord ( RefundID INT PRIMARY KEY IDENTITY(1,1), SaleID INT NOT NULL, RefundQuantity INT NOT NULL CHECK (RefundQuantity > 0), RefundAmount DECIMAL(10,2) NOT NULL, RefundTime DATETIME DEFAULT GETDATE(), Reason NVARCHAR(255), CONSTRAINT FK_RefundRecord_TicketSale FOREIGN KEY (SaleID) REFERENCES TicketSale(SaleID) ); GO -- 创建操作日志表 CREATE TABLE AuditLog ( LogID INT PRIMARY KEY IDENTITY(1,1), UserID INT NOT NULL, ActionType NVARCHAR(20) NOT NULL, ActionDetails NVARCHAR(500) NOT NULL, ActionTime DATETIME DEFAULT GETDATE(), CONSTRAINT FK_AuditLog_Salesperson FOREIGN KEY (UserID) REFERENCES Salesperson(SalespersonID) ); GO -- 创建索引优化查询性能 CREATE INDEX IX_TicketSale_SaleTime ON TicketSale(SaleTime); CREATE INDEX IX_TicketSale_Salesperson ON TicketSale(SalespersonID); CREATE INDEX IX_TicketSale_Type ON TicketSale(TypeID); CREATE INDEX IX_RefundRecord_SaleID ON RefundRecord(SaleID); CREATE INDEX IX_AuditLog_ActionTime ON AuditLog(ActionTime); GO -- 存储过程1: 统计指定日期的门票销售情况 CREATE PROCEDURE sp_DailySalesSummary @TargetDate DATE AS BEGIN SELECT TT.TypeName AS 票种类型, SUM(TS.Quantity) AS 总票数, SUM(TS.TotalAmount) AS 总收入 FROM TicketSale TS JOIN TicketType TT ON TS.TypeID = TT.TypeID WHERE CAST(TS.SaleTime AS DATE) = @TargetDate AND TS.IsRefund = 0 GROUP BY TT.TypeName ORDER BY 总收入 DESC; END; GO -- 存储过程2: 统计指定月份的门票销售情况 CREATE PROCEDURE sp_MonthlySalesSummary @Year INT, @Month INT AS BEGIN SELECT TT.TypeName AS 票种类型, SUM(TS.Quantity) AS 总票数, SUM(TS.TotalAmount) AS 总收入 FROM TicketSale TS JOIN TicketType TT ON TS.TypeID = TT.TypeID WHERE YEAR(TS.SaleTime) = @Year AND MONTH(TS.SaleTime) = @Month AND TS.IsRefund = 0 GROUP BY TT.TypeName ORDER BY 总收入 DESC; END; GO -- 存储过程3: 统计指定日期各种价格的门票销售情况 CREATE PROCEDURE sp_DailySalesByPrice @TargetDate DATE AS BEGIN SELECT TS.UnitPrice AS 销售单价, COUNT(TS.SaleID) AS 销售笔数, SUM(TS.Quantity) AS 总票数, SUM(TS.TotalAmount) AS 总收入 FROM TicketSale TS WHERE CAST(TS.SaleTime AS DATE) = @TargetDate AND TS.IsRefund = 0 GROUP BY TS.UnitPrice ORDER BY TS.UnitPrice DESC; END; GO -- 存储过程4: 统计指定营业员指定日期的收费情况 CREATE PROCEDURE sp_SalespersonDailySummary @SalespersonID INT, @TargetDate DATE AS BEGIN SELECT SP.Name AS 营业员姓名, COUNT(TS.SaleID) AS 交易笔数, SUM(TS.Quantity) AS 总票数, SUM(TS.TotalAmount) AS 总收入 FROM TicketSale TS JOIN Salesperson SP ON TS.SalespersonID = SP.SalespersonID WHERE SP.SalespersonID = @SalespersonID AND CAST(TS.SaleTime AS DATE) = @TargetDate AND TS.IsRefund = 0 GROUP BY SP.Name; -- 详细交易列表 SELECT TT.TypeName AS 票种类型, TS.Quantity AS 数量, TS.UnitPrice AS 单价, TS.TotalAmount AS 金额, FORMAT(TS.SaleTime, 'HH:mm:ss') AS 销售时间 FROM TicketSale TS JOIN TicketType TT ON TS.TypeID = TT.TypeID WHERE TS.SalespersonID = @SalespersonID AND CAST(TS.SaleTime AS DATE) = @TargetDate AND TS.IsRefund = 0 ORDER BY TS.SaleTime DESC; END; GO -- 存储过程5: 处理门票销售 CREATE PROCEDURE sp_ProcessTicketSale @TypeID INT, @SalespersonID INT, @Quantity INT, @SaleID INT OUTPUT AS BEGIN BEGIN TRY BEGIN TRANSACTION; DECLARE @UnitPrice DECIMAL(10,2); DECLARE @TotalAmount DECIMAL(10,2); -- 获取票价信息 SELECT @UnitPrice = BasePrice * DiscountRate FROM TicketType WHERE TypeID = @TypeID; IF @UnitPrice IS NULL RAISERROR('无效的票种ID', 16, 1); -- 计算总金额 SET @TotalAmount = @UnitPrice * @Quantity; -- 插入销售记录 INSERT INTO TicketSale ( TypeID, SalespersonID, Quantity, UnitPrice, TotalAmount ) VALUES ( @TypeID, @SalespersonID, @Quantity, @UnitPrice, @TotalAmount ); SET @SaleID = SCOPE_IDENTITY(); -- 记录操作日志 INSERT INTO AuditLog (UserID, ActionType, ActionDetails) VALUES ( @SalespersonID, 'SALE', CONCAT('销售门票: 票种ID=', @TypeID, ', 数量=', @Quantity, ', 金额=', @TotalAmount) ); COMMIT TRANSACTION; END TRY BEGIN CATCH ROLLBACK TRANSACTION; DECLARE @ErrorMessage NVARCHAR(4000) = ERROR_MESSAGE(); RAISERROR(@ErrorMessage, 16, 1); END CATCH END; GO -- 存储过程6: 处理门票退票 CREATE PROCEDURE sp_ProcessTicketRefund @SaleID INT, @RefundQuantity INT, @Reason NVARCHAR(255), @RefundID INT OUTPUT AS BEGIN BEGIN TRY BEGIN TRANSACTION; DECLARE @MaxQuantity INT; DECLARE @UnitPrice DECIMAL(10,2); DECLARE @RefundAmount DECIMAL(10,2); -- 获取原始销售信息 SELECT @MaxQuantity = Quantity, @UnitPrice = UnitPrice FROM TicketSale WHERE SaleID = @SaleID; IF @MaxQuantity IS NULL RAISERROR('无效的销售ID', 16, 1); -- 检查可退票数量 DECLARE @AlreadyRefunded INT = 0; SELECT @AlreadyRefunded = ISNULL(SUM(RefundQuantity), 0) FROM RefundRecord WHERE SaleID = @SaleID; DECLARE @Remaining INT = @MaxQuantity - @AlreadyRefunded; IF @RefundQuantity > @Remaining RAISERROR('退票数量超过可退数量', 16, 1); -- 计算退款金额 SET @RefundAmount = @UnitPrice * @RefundQuantity; -- 插入退票记录 INSERT INTO RefundRecord ( SaleID, RefundQuantity, RefundAmount, Reason ) VALUES ( @SaleID, @RefundQuantity, @RefundAmount, @Reason ); SET @RefundID = SCOPE_IDENTITY(); -- 更新销售记录状态 IF @RefundQuantity = @Remaining UPDATE TicketSale SET IsRefund = 1 WHERE SaleID = @SaleID; -- 记录操作日志 DECLARE @SalespersonID INT = (SELECT SalespersonID FROM TicketSale WHERE SaleID = @SaleID); INSERT INTO AuditLog (UserID, ActionType, ActionDetails) VALUES ( @SalespersonID, 'REFUND', CONCAT('退票处理: 销售ID=', @SaleID, ', 数量=', @RefundQuantity, ', 金额=', @RefundAmount) ); COMMIT TRANSACTION; END TRY BEGIN CATCH ROLLBACK TRANSACTION; DECLARE @ErrorMessage NVARCHAR(4000) = ERROR_MESSAGE(); RAISERROR(@ErrorMessage, 16, 1); END CATCH END; GO -- 存储过程7: 获取所有有效票种 CREATE PROCEDURE sp_GetActiveTicketTypes AS BEGIN SELECT TypeID, TypeName, BasePrice, DiscountRate, ROUND(BasePrice * DiscountRate, 2) AS ActualPrice, Description FROM TicketType ORDER BY TypeName; END; GO -- 存储过程8: 添加新票种 CREATE PROCEDURE sp_AddTicketType @TypeName NVARCHAR(20), @BasePrice DECIMAL(10,2), @DiscountRate DECIMAL(5,2) = 1.0, @Description NVARCHAR(255) = NULL AS BEGIN IF NOT @TypeName IN ('老年', '小孩', '成人', '团体') RAISERROR('无效的票种名称', 16, 1); INSERT INTO TicketType ( TypeName, BasePrice, DiscountRate, Description ) VALUES ( @TypeName, @BasePrice, @DiscountRate, @Description ); RETURN SCOPE_IDENTITY(); END; GO -- 存储过程9: 营业员登录验证 CREATE PROCEDURE sp_VerifySalespersonLogin @EmployeeID NVARCHAR(20), @Password NVARCHAR(100), @IsValid BIT OUTPUT AS BEGIN SET @IsValid = 0; IF EXISTS ( SELECT 1 FROM Salesperson WHERE EmployeeID = @EmployeeID AND Password = @Password AND IsActive = 1 ) BEGIN SET @IsValid = 1; -- 更新最后登录时间 UPDATE Salesperson SET LastLogin = GETDATE() WHERE EmployeeID = @EmployeeID; END END; GO -- 存储过程10: 月度销售趋势分析 CREATE PROCEDURE sp_MonthlySalesTrend @Year INT AS BEGIN SELECT MONTH(SaleTime) AS 月份, SUM(TotalAmount) AS 月收入, COUNT(SaleID) AS 交易笔数, SUM(Quantity) AS 总票数 FROM TicketSale WHERE YEAR(SaleTime) = @Year AND IsRefund = 0 GROUP BY MONTH(SaleTime) ORDER BY 月份; END; GO -- 创建触发器: 更新票种时自动更新更新时间戳 CREATE TRIGGER trg_UpdateTicketTypeTimestamp ON TicketType AFTER UPDATE AS BEGIN UPDATE TicketType SET UpdatedAt = GETDATE() WHERE TypeID IN (SELECT TypeID FROM inserted); END; GO -- 创建触发器: 退票后更新销售记录状态 CREATE TRIGGER trg_AfterRefund ON RefundRecord AFTER INSERT AS BEGIN SET NOCOUNT ON; UPDATE TS SET TS.IsRefund = CASE WHEN (TS.Quantity - i.RefundQuantity) <= 0 THEN 1 ELSE 0 END FROM TicketSale TS INNER JOIN inserted i ON TS.SaleID = i.SaleID; END; GO -- 创建视图: 销售明细视图 CREATE VIEW vw_SaleDetails AS SELECT TS.SaleID, TS.SaleTime, SP.Name AS SalespersonName, TT.TypeName, TS.Quantity, TS.UnitPrice, TS.TotalAmount, CASE WHEN TS.IsRefund = 1 THEN '已退票' ELSE '有效' END AS Status, ISNULL(RR.RefundQuantity, 0) AS RefundedQuantity FROM TicketSale TS JOIN Salesperson SP ON TS.SalespersonID = SP.SalespersonID JOIN TicketType TT ON TS.TypeID = TT.TypeID LEFT JOIN ( SELECT SaleID, SUM(RefundQuantity) AS RefundQuantity FROM RefundRecord GROUP BY SaleID ) RR ON TS.SaleID = RR.SaleID; GO -- 插入初始数据 INSERT INTO TicketType (TypeName, BasePrice, DiscountRate, Description) VALUES ('成人', 100.00, 1.00, '标准成人票'), ('小孩', 50.00, 1.00, '1.2-1.5米儿童票'), ('老年', 60.00, 1.00, '65岁以上老人票'), ('团体', 80.00, 0.90, '10人及以上团体票'); GO INSERT INTO Salesperson (EmployeeID, Name, Password, Role) VALUES ('admin001', '管理员', 'admin123', 'admin'), ('staff001', '张三', 'staff123', 'staff'), ('staff002', '李四', 'staff456', 'staff'); GO -- 插入示例销售记录 DECLARE @SaleID INT; EXEC sp_ProcessTicketSale 1, 2, 2, @SaleID OUTPUT; EXEC sp_ProcessTicketSale 2, 2, 1, @SaleID OUTPUT; EXEC sp_ProcessTicketSale 3, 3, 3, @SaleID OUTPUT; EXEC sp_ProcessTicketSale 4, 3, 10, @SaleID OUTPUT; GO -- 插入示例退票记录 DECLARE @RefundID INT; EXEC sp_ProcessTicketRefund 1, 1, '行程变更', @RefundID OUTPUT; GO数据库编写完后给出下一个步骤,使用后端使用Java,前端使用html
最新发布
06-10
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值