海山数据库(He3DB)源码详解:海山MySQL redo日志-写入过程

# 一、redo log block

设计InnoDB时为了更好的进行系统奔溃恢复,将通过mtr生成的redo日志放在大小为512字节的页中。为了和表空间中的页做区别,于是把用来存储redo日志的页称为block。一个redo log block的示意图如下:

在这里插入图片描述

真正的redo日志都是存储到占用496字节大小的log block body中。另外,log block header占12个字节,log block trailer占4个字节,存储的是一些管理信息。

其中,log block header中包含以下属性字段:

  • LOG_BLOCK_HDR_NO(4B):每一个block都有一个大于0的唯一标号,该属性就表示该标号值。
  • LOG_BLOCK_HDR_DATA_LEN(2B):表示block中已经使用了多少字节,初始值为12。随着往block中写入的redo日志越来也多,该值也跟着增长。如果log block body已经被全部写满,那么值被设置为512。
  • LOG_BLOCK_FIRST_REC_GROUP(2B):一条redo日志也可以称之为一条redo日志记录,一个mtr会生产多条redo日志记录,这些redo日志记录被称之为一个redo日志记录组(redo log record group)。LOG_BLOCK_FIRST_REC_GROUP就代表该block中第一个mtr生成的redo日志记录组的偏移量,即这个block里第一个mtr生成的第一条redo日志的偏移量。
  • LOG_BLOCK_CHECKPOINT_NO(4B):表示checkpoint的序号。

log block trailer中包含的属性字段为:

  • LOG_BLOCK_CHECKSUM(4B):表示block的校验值,用于正确性校验。

二、redo日志缓冲区

与为了解决磁盘速度过慢的问题而引入了Buffer Pool的思想类似,写入redo日志时也不能直接直接写到磁盘上。实际上在服务器启动时就向操作系统申请了一大⽚称之为redo log buffer的连续内存空间,即redo日志缓冲区,也可以简称为log buffer。这⽚内存空间被划分成若干个连续的redo log block,如图所示:

在这里插入图片描述

三、redo日志写入log buffer

log buffer中写入redo日志的过程是顺序的,也就是先往前边的block中写,当该block的空闲空间用完之后再往下一个block中写。因此,当往log buffer中写入redo日志时,第一个遇到的问题就是应该写在哪个block的哪个偏移量处,所以InnoDB的特意提供了一个称之为buf_free的全局变量,该变量指明后续写入的redo日志应该写入到log buffer中的哪个位置。

在这里插入图片描述

由于一个mtr执行过程中可能产生若干条redo日志,这些redo日志是一个不可分割的组,所以其实并不是每生成一条redo日志,就将其插入到log buffer中,而是每个mtr运行过程中产生的日志先暂时存到一个地方,当该mtr结束的时候,将过程中产生的一组redo日志再全部复制到log buffer中。

四、源码解析

4.1 log buffer结构体

/** Redo log buffer */
struct log_t{
   
   
	char		pad1[CACHE_LINE_SIZE];
					/*!< Padding to prevent other memory
					update hotspots from residing on the
					same memory cache line */
	lsn_t		lsn;		/*!< log sequence number */
	ulint		buf_free;	/*!< first free offset within the log
					buffer in use */
	byte*		buf_ptr;	/*!< unaligned log buffer, which should
					be of double of buf_size */
	byte*		buf;		/*!< log buffer currently in use;
					this could point to either the first
					half of the aligned(buf_ptr) or the
					second half in turns, so that log
					write/flush to disk don't block
					concurrent mtrs which will write
					log to this buffer */
	bool		first_in_use;	/*!< true if buf points to the first
					half of the aligned(buf_ptr), false
					if the second half */
	ulint		buf_size;	/*!< log buffer size of each in bytes */
	ulint		max_buf_free;	/*!< recommended maximum value of
					buf_free for the buffer in use, after
					which the buffer is flushed */
	bool		check_flush_or_checkpoint;
					/*!< this is set when there may
					be need to flush the log buffer, or
					preflush buffer pool pages, or make
					a checkpoint; this MUST be TRUE when
					lsn - last_checkpoint_lsn >
					max_checkpoint_age; this flag is
					peeked at by log_free_check(), which
					does not reserve the log mutex */
	UT_LIST_BASE_NODE_T(log_group_t)
			log_groups;	/*!< log groups */

#ifndef UNIV_HOTBACKUP
	/** The fields involved in the log buffer flush @{ */

	ulint		buf_next_to_write;/*!< first offset in the log buffer
					where the byte content may not exist
					written to file, e.g., the start
					offset of a log record catenated
					later; this is advanced when a flush
					operation is completed to all the log
					groups */
	volatile bool	is_extending;	/*!< this is set to true during extend
					the log buffer size */
	lsn_t		write_lsn;	/*!< last written lsn */
	lsn_t		current_flush_lsn;/*!< end lsn for the current running
					write + flush operation */
	lsn_t		flushed_to_disk_lsn;
					/*!< how far we have written the log
					AND flushed to disk */
	ulint		n_pending_flushes;/*!< number of currently
					pending flushes; incrementing is
					protected by the log mutex;
					may be decremented between
					resetting and setting flush_event */
	os_event_t	flush_event;	/*!< this event is in the reset state
					when a flush is running; a thread
					should wait for this without
					owning the log mutex, but NOTE that
					to set this event, the
					thread MUST own the log mutex! */
	ulint		n_log_ios;	/*!< number of log i/os initiated thus
					far */
	ulint		n_log_ios_old;	/*!< number of log i/o's at the
					previous printout */
	time_t		last_printout_time;/*!< when log_print was last time
					called */
	/* @} */

	/** Fields involved in checkpoints @{ */
	lsn_t		log_group_capacity; /*!< capacity of the log group; if
					the checkpoint age exceeds this, it is
					a serious error because it is possible
					we will then overwrite log and spoil
					crash recovery */
	lsn_t		max_modified_age_async;
					/*!< when this recommended
					value for lsn -
					buf_pool_get_oldest_modification()
					is exceeded, we start an
					asynchronous preflush of pool pages */
	lsn_t		max_modified_age_sync;
					/*!< when this recommended
					value for lsn -
					buf_pool_get_oldest_modification()
					is exceeded, we start a
					synchronous preflush of pool pages */
	lsn_t		max_checkpoint_age_async;
					/*!< when this checkpoint age
					is exceeded we start an
					asynchronous writing of a new
					checkpoint */
	lsn_t		max_checkpoint_age;
					/*!< this is the maximum allowed value
					for lsn - last_checkpoint_lsn when a
					new query step is started */
	ib_uint64_t	next_checkpoint_no;
					/*!< next checkpoint number */
	lsn_t		last_checkpoint_lsn;
					/*!< latest checkpoint lsn */
	
根据日志获取服务不可用阶段Build 湘雅三GCP(生产环境) - dockerbuild - Default Job #83 (GCPXY3-DOC-JOB1-83) started building on agent Local Agent1, bamboo version: 8.1.1 simple 12-Aug-2025 17:57:36 Local Agent1 simple 12-Aug-2025 17:57:36 Build working directory is /var/atlassian/application-data/bamboo/local-working-dir/622593/GCPXY3-DOC-JOB1 simple 12-Aug-2025 17:57:36 Executing build 湘雅三GCP(生产环境) - dockerbuild - Default Job #83 (GCPXY3-DOC-JOB1-83) simple 12-Aug-2025 17:57:36 Running pre-build action: VCS Version Collector simple 12-Aug-2025 17:57:36 Running pre-build action: Build Log Labeller Pre Build Action command 12-Aug-2025 17:57:36 Substituting variable: ${bamboo.prod_xy3} with 119.91.104.191 simple 12-Aug-2025 17:57:36 Starting task &#39;SSH Task&#39; of type &#39;com.atlassian.bamboo.plugins.bamboo-scp-plugin:sshtask&#39; simple 12-Aug-2025 17:57:36 Connecting to 119.91.104.191 on port: 2022 simple 12-Aug-2025 17:57:37 Executing [ simple 12-Aug-2025 17:57:37 cd /opt/xy3/binaries/ simple 12-Aug-2025 17:57:37 docker-compose -f docker-compose-web-prod.yml up -d --build --force-recreate ghc-web simple 12-Aug-2025 17:57:37 docker-compose -f docker-compose-prod.yml up -d --build --force-recreate ctms-auth-prod simple 12-Aug-2025 17:57:37 docker-compose -f docker-compose-prod.yml up -d --build --force-recreate ctms-gateway-prod simple 12-Aug-2025 17:57:37 docker-compose -f docker-compose-prod.yml up -d --build --force-recreate ctms-system-prod simple 12-Aug-2025 17:57:37 docker-compose -f docker-compose-prod.yml up -d --build --force-recreate ctms-irbs-prod simple 12-Aug-2025 17:57:37 simple 12-Aug-2025 17:57:37 docker system prune -f simple 12-Aug-2025 17:57:37 ] build 12-Aug-2025 18:00:09 Step 1/6 : FROM java:8 build 12-Aug-2025 18:00:09 ---> d23bdf5b1b1b build 12-Aug-2025 18:00:09 Step 2/6 : VOLUME /tmp build 12-Aug-2025 18:00:09 ---> Using cache build 12-Aug-2025 18:00:09 ---> 65ebb77cad09 build 12-Aug-2025 18:00:09 Step 3/6 : ARG version build 12-Aug-2025 18:00:09 ---> Using cache build 12-Aug-2025 18:00:09 ---> 227aa09399d2 build 12-Aug-2025 18:00:09 Step 4/6 : ARG app build 12-Aug-2025 18:00:09 ---> Using cache build 12-Aug-2025 18:00:09 ---> 81b3499de399 build 12-Aug-2025 18:00:09 Step 5/6 : COPY services/${version}/lib/${app}/ ./lib/ build 12-Aug-2025 18:00:09 ---> Using cache build 12-Aug-2025 18:00:09 ---> 1e7dd4b95377 build 12-Aug-2025 18:00:09 Step 6/6 : ADD services/${version}/${app}.jar ${app}.jar build 12-Aug-2025 18:00:15 ---> cab7592fa7f9 build 12-Aug-2025 18:00:15 build 12-Aug-2025 18:00:17 Successfully built cab7592fa7f9 build 12-Aug-2025 18:00:17 Successfully tagged ctms-auth-prod:1.0.0 build 12-Aug-2025 18:02:14 Step 1/6 : FROM java:8 build 12-Aug-2025 18:02:15 ---> d23bdf5b1b1b build 12-Aug-2025 18:02:15 Step 2/6 : VOLUME /tmp build 12-Aug-2025 18:02:15 ---> Using cache build 12-Aug-2025 18:02:15 ---> 65ebb77cad09 build 12-Aug-2025 18:02:15 Step 3/6 : ARG version build 12-Aug-2025 18:02:15 ---> Using cache build 12-Aug-2025 18:02:15 ---> 227aa09399d2 build 12-Aug-2025 18:02:15 Step 4/6 : ARG app build 12-Aug-2025 18:02:15 ---> Using cache build 12-Aug-2025 18:02:15 ---> 81b3499de399 build 12-Aug-2025 18:02:15 Step 5/6 : COPY services/${version}/lib/${app}/ ./lib/ build 12-Aug-2025 18:02:16 ---> Using cache build 12-Aug-2025 18:02:16 ---> 9036f99d9220 build 12-Aug-2025 18:02:16 Step 6/6 : ADD services/${version}/${app}.jar ${app}.jar build 12-Aug-2025 18:02:29 ---> 7cdc7da77ff7 build 12-Aug-2025 18:02:29 build 12-Aug-2025 18:02:30 Successfully built 7cdc7da77ff7 build 12-Aug-2025 18:02:30 Successfully tagged ctms-gateway-prod:1.0.0 build 12-Aug-2025 18:04:17 Step 1/6 : FROM java:8 build 12-Aug-2025 18:04:18 ---> d23bdf5b1b1b build 12-Aug-2025 18:04:18 Step 2/6 : VOLUME /tmp build 12-Aug-2025 18:04:18 ---> Using cache build 12-Aug-2025 18:04:18 ---> 65ebb77cad09 build 12-Aug-2025 18:04:18 Step 3/6 : ARG version build 12-Aug-2025 18:04:18 ---> Using cache build 12-Aug-2025 18:04:18 ---> 227aa09399d2 build 12-Aug-2025 18:04:18 Step 4/6 : ARG app build 12-Aug-2025 18:04:18 ---> Using cache build 12-Aug-2025 18:04:18 ---> 81b3499de399 build 12-Aug-2025 18:04:18 Step 5/6 : COPY services/${version}/lib/${app}/ ./lib/ build 12-Aug-2025 18:04:18 ---> Using cache build 12-Aug-2025 18:04:18 ---> aa994c85a058 build 12-Aug-2025 18:04:18 Step 6/6 : ADD services/${version}/${app}.jar ${app}.jar build 12-Aug-2025 18:04:27 ---> 52ea93d68a7c build 12-Aug-2025 18:04:27 build 12-Aug-2025 18:04:28 Successfully built 52ea93d68a7c build 12-Aug-2025 18:04:28 Successfully tagged ctms-system-prod:1.0.0 build 12-Aug-2025 18:06:09 Step 1/6 : FROM java:8 build 12-Aug-2025 18:06:10 ---> d23bdf5b1b1b build 12-Aug-2025 18:06:10 Step 2/6 : VOLUME /tmp build 12-Aug-2025 18:06:10 ---> Using cache build 12-Aug-2025 18:06:10 ---> 65ebb77cad09 build 12-Aug-2025 18:06:10 Step 3/6 : ARG version build 12-Aug-2025 18:06:10 ---> Using cache build 12-Aug-2025 18:06:10 ---> 227aa09399d2 build 12-Aug-2025 18:06:10 Step 4/6 : ARG app build 12-Aug-2025 18:06:11 ---> Using cache build 12-Aug-2025 18:06:11 ---> 81b3499de399 build 12-Aug-2025 18:06:11 Step 5/6 : COPY services/${version}/lib/${app}/ ./lib/ build 12-Aug-2025 18:06:12 ---> Using cache build 12-Aug-2025 18:06:12 ---> 06af45c2f79d build 12-Aug-2025 18:06:12 Step 6/6 : ADD services/${version}/${app}.jar ${app}.jar build 12-Aug-2025 18:06:23 ---> d28fed9c95a3 build 12-Aug-2025 18:06:23 build 12-Aug-2025 18:06:24 Successfully built d28fed9c95a3 build 12-Aug-2025 18:06:24 Successfully tagged ctms-irbs-prod:1.0.0 build 12-Aug-2025 18:06:26 Deleted Images: build 12-Aug-2025 18:06:26 deleted: sha256:b5ba06737e2e551082b8096fbc83ce3675dac3ccfd413b9fe1b15121fbf97cf9 build 12-Aug-2025 18:06:26 deleted: sha256:35c76628a0ea6c92bf3f1e23a65d39b65cb9cfaf83af57df5293573bff5b944f build 12-Aug-2025 18:06:26 deleted: sha256:904ebc147f8ad93396e7ee55cb978baf913c7a9934b2249d22da7c798f15f5c1 build 12-Aug-2025 18:06:26 deleted: sha256:6587fc1c0e72fe65af594441b34436d18df5546688aa564f1419dfa50b022112 build 12-Aug-2025 18:06:26 deleted: sha256:1c8813bff1f6ce731675cf4229211dd4bf3421e7548792cf7657591a970494ac build 12-Aug-2025 18:06:26 deleted: sha256:16b2e7700101600360923d8b05101ff56e06d8da0928ef41edbf2b3f7a68c30c build 12-Aug-2025 18:06:26 deleted: sha256:943137eae2d76ad2baa2753da688278509d17f3b13c36b819dd66126db3db594 build 12-Aug-2025 18:06:26 deleted: sha256:ad52f9ba3affa27cdfc761c46ee7233344dae05480051f6f00da31088f608174 build 12-Aug-2025 18:06:26 build 12-Aug-2025 18:06:26 Total reclaimed space: 24.83MB error 12-Aug-2025 18:06:26 Found orphan containers (ghc-nacos, ctms-irbs-prod, ghc-minio, ctms-auth-prod, ctms-system-prod, ctms-gateway-prod, ghc-mysql, ghc-mysql2, ghc-redis) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. error 12-Aug-2025 18:06:26 Recreating ghc-web ... error 12-Aug-2025 18:06:26  error 12-Aug-2025 18:06:26 Recreating ghc-web ... done error 12-Aug-2025 18:06:26 Found orphan containers (ghc-mysql, ghc-minio, ghc-web, ghc-mysql2, ghc-nacos, ghc-redis) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. error 12-Aug-2025 18:06:26 Building ctms-auth-prod error 12-Aug-2025 18:06:26 Recreating ctms-auth-prod ... error 12-Aug-2025 18:06:26  error 12-Aug-2025 18:06:26 Recreating ctms-auth-prod ... done error 12-Aug-2025 18:06:26 Found orphan containers (ghc-mysql, ghc-web, ghc-mysql2, ghc-nacos, ghc-minio, ghc-redis) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. error 12-Aug-2025 18:06:26 Building ctms-gateway-prod error 12-Aug-2025 18:06:26 Recreating ctms-gateway-prod ... error 12-Aug-2025 18:06:26  error 12-Aug-2025 18:06:26 Recreating ctms-gateway-prod ... done error 12-Aug-2025 18:06:26 Found orphan containers (ghc-redis, ghc-nacos, ghc-web, ghc-mysql2, ghc-mysql, ghc-minio) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. error 12-Aug-2025 18:06:26 Building ctms-system-prod error 12-Aug-2025 18:06:26 Recreating ctms-system-prod ... error 12-Aug-2025 18:06:26  error 12-Aug-2025 18:06:26 Recreating ctms-system-prod ... done error 12-Aug-2025 18:06:26 Found orphan containers (ghc-mysql2, ghc-redis, ghc-nacos, ghc-mysql, ghc-minio, ghc-web) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. error 12-Aug-2025 18:06:26 Building ctms-irbs-prod error 12-Aug-2025 18:06:26 Recreating ctms-irbs-prod ... error 12-Aug-2025 18:06:26  error 12-Aug-2025 18:06:26 Recreating ctms-irbs-prod ... done error 12-Aug-2025 18:06:26  simple 12-Aug-2025 18:06:26 [ simple 12-Aug-2025 18:06:26 cd /opt/xy3/binaries/ simple 12-Aug-2025 18:06:26 docker-compose -f docker-compose-web-prod.yml up -d --build --force-recreate ghc-web simple 12-Aug-2025 18:06:26 docker-compose -f docker-compose-prod.yml up -d --build --force-recreate ctms-auth-prod simple 12-Aug-2025 18:06:26 docker-compose -f docker-compose-prod.yml up -d --build --force-recreate ctms-gateway-prod simple 12-Aug-2025 18:06:26 docker-compose -f docker-compose-prod.yml up -d --build --force-recreate ctms-system-prod simple 12-Aug-2025 18:06:26 docker-compose -f docker-compose-prod.yml up -d --build --force-recreate ctms-irbs-prod simple 12-Aug-2025 18:06:26 simple 12-Aug-2025 18:06:26 docker system prune -f simple 12-Aug-2025 18:06:26 ] has finished. simple 12-Aug-2025 18:06:26 Result: exit code = 0 simple 12-Aug-2025 18:06:26 Finished task &#39;SSH Task&#39; with result: Success simple 12-Aug-2025 18:06:26 Running post build plugin &#39;NCover Results Collector&#39; simple 12-Aug-2025 18:06:26 Running post build plugin &#39;Artifact Copier&#39; simple 12-Aug-2025 18:06:26 Running post build plugin &#39;npm Cache Cleanup&#39; simple 12-Aug-2025 18:06:26 Running post build plugin &#39;Build Results Label Collector&#39; simple 12-Aug-2025 18:06:26 Running post build plugin &#39;Clover Results Collector&#39; simple 12-Aug-2025 18:06:26 Running post build plugin &#39;Docker Container Cleanup&#39; simple 12-Aug-2025 18:06:26 Finalising the build... simple 12-Aug-2025 18:06:26 Stopping timer. simple 12-Aug-2025 18:06:26 Build GCPXY3-DOC-JOB1-83 completed. simple 12-Aug-2025 18:06:26 Running on server: post build plugin &#39;NCover Results Collector&#39; simple 12-Aug-2025 18:06:26 Running on server: post build plugin &#39;Build Hanging Detection Configuration&#39; simple 12-Aug-2025 18:06:26 Running on server: post build plugin &#39;Build Labeller&#39; simple 12-Aug-2025 18:06:26 Running on server: post build plugin &#39;Clover Delta Calculator&#39; simple 12-Aug-2025 18:06:26 Running on server: post build plugin &#39;Maven Dependencies Postprocessor&#39; simple 12-Aug-2025 18:06:26 All post build plugins have finished simple 12-Aug-2025 18:06:26 Generating build results summary... simple 12-Aug-2025 18:06:26 Saving build results to disk... simple 12-Aug-2025 18:06:26 Store variable context... simple 12-Aug-2025 18:06:26 Indexing build results... simple 12-Aug-2025 18:06:26 Finished building GCPXY3-DOC-JOB1-83.
最新发布
08-14
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值