Fast Project Rollbacks

 

Fast Project Rollbacks

The projects that I work on require both deployment and rollback scripts. Using a database backup as a rollback strategy is not acceptable for a number of reasons.

  • Downtime needs to be measured in minutes and the restore would take hours.
  • The project normally affects a portion of the databases so rollbacks are intended to affect the objects and data that have been deployed and not any other objects or data.
  • Visibility of the rollback steps is needed as an audit log.

I have a particular approach to scripting up the rollback of database projects that I should like to share with you.

Goals for creating rollback scripts

Code readability

All code has to be peer-reviewed and the effort involved in a thorough peer review is not always appreciated. Indeed Cisco used to publish recommendations for peer reviews that include recommendations on the following

  • The amount of code that could be expected to be peer reviewed per session.
  • The amount of time in minutes allowed in any one peer review session before a break was needed.

The figures for both the volume of code and the time spent are likely to be far lower than you would expect therefore anything you can do to make the code easier to read is going to aid the peer review process.

Remember a peer review should not be regarded as an obstacle to overcome but as an insurance policy to ensure that it won't be your code that is to blame for a production incident.

If your code is readable then the peer review can be achieved quicker and any bugs should be easier to spot.

Code readability leads to my second goal

Code Maintainability

I don't code on the fly. I put quite a bit of time and effort into my design but if I can use the analogy of the battle plan then no battle plan survives engagement with the enemy.

Things that made perfect sense in the design stage might not be practical in the real world. My code is probably going to change so it must be easy to change.

Code Durability

If a rollback script runs more than once then it should do no harm and produce no error messages. I am only concerned with rollback scripts here but the same comment also applies to deployment scripts. Ideally the both deployment and rollback scripts should produce status messages to act as a log of what has taken place.

Most of my deployments start in the very early hours of the morning when I'm fully decaffeinated so any unexpected error messages are unwelcome.

Speed of execution

Time is money. If your database is unavailable then it is costing you money. In some cases using a backup as a rollback strategy simply isn't financially feasible except as a method of last resort.

Having a script that can execute in seconds, or at the worst minutes is far better than a restore of a terabyte database that may take hours.

Script headers

I use script headers for three purposes.

  • To provider brief notes on what the script is supposed to do, prerequisites, special actions
  • To generate dynamic status messages showing the server, time of deployment, the account running the script.
  • To set session settings such as XACT_ABORT and NOCOUNT ON.

Consider the code below.

SET XACT_ABORT,NOCOUNT ON
---------------------------------------------------
USE AW_TARGET
GO
---------------------------------------------------
PRINT '---------------------------------------------------'
DECLARE 
	@CRLF CHAR(2) ,-- Carriage return and line feed
	@Deployer SYSNAME, -- The account running the script
	@DeploymentDate CHAR(20), -- The date and time of execution
	@DBName SYSNAME -- The database name to which the action applies.
SET @CRLF=CHAR(13)+CHAR(10)
SET @Deployer = SUSER_SNAME()
SET @DeploymentDate = convert(CHAR(20),getdate(),120)
SET @DBName = DB_NAME()
RAISERROR('DEPLOYMENT SERVER: %s%sDEPLOYER: %s%sDeployment Date: %s%sDatabase: %s',10,1,@@SERVERNAME,@CRLF,@Deployer,@CRLF, @DeploymentDate,@CRLF, @DBName)
PRINT '---------------------------------------------------'

The first line simply switches the following two session options on.

Session option

Action when ON

XACT_ABORT

Any action resulting in an error will rollback the transaction in which the error occurred.

NOCOUNT

Suppresses the (n row(s) affected) message and makes the audit messages easier to read.

In this case the AW_TARGET database is simply a copy of AdventureWorks

The remaining line cause the following message to be displayed

---------------------------------------------------
DEPLOYMENT SERVER: DBDevelopment01
DEPLOYER: Development01/David
Deployment Date: 2008-09-07 15:54:33 
Database: AW_Target
---------------------------------------------------

Removing views

The following code may look familiar

IF  EXISTS (INFORMATION_SCHEMA.VIEWS WHERE TABLE_SCHEMA='Production' AND TABLE_NAME='vProductAndDescription')
	DROP VIEW Production.vProductAndDescription
GO

Mechanically there is nothing wrong with the above but for removing multiple objects the repetitive nature of the code makes it easy to miss errors.

The following code shows an alternative method of removing a set list of views


DECLARE @SQL VARCHAR(MAX) , @CRLF CHAR(2)
SET @CRLF = CHAR(13)+CHAR(10)
SELECT @SQL=COALESCE(@SQL+';'+@CRLF,'') 
+ 'DROP VIEW '
+ QUOTENAME(TABLE_SCHEMA)
+ '.'
+ QUOTENAME(TABLE_NAME)
FROM INFORMATION_SCHEMA.[VIEWS]
WHERE 
	TABLE_SCHEMA='Production'
AND TABLE_NAME IN (
	'vProductAndDescription',
	'vProductModelCatalogDescription',
	'vProductModelInstructions'
)
PRINT @SQL
--EXEC (@SQL)
GO

Although it may seem like overkill for three simple views the process comes into its own when the view list gets longer. The view list acts as a manifest that can be checked against the deployment script or release notes.

To prevent accidents while experimenting I have commented out the EXEC statement but when run the script will produce the following.

---------------------------------------------------
DEPLOYMENT SERVER: D_POOLE
DEPLOYER: D_POOLE/David
Deployment Date: 2008-09-07 16:05:05 
Database: AW_Target
---------------------------------------------------
DROP VIEW [Production].[vProductAndDescription];
DROP VIEW [Production].[vProductModelCatalogDescription];
DROP VIEW [Production].[vProductModelInstructions]

Removing Stored Procedures

The method for removing stored procedures is almost identical differing only in the INFORMATION_SCHEMA view and associated field names.


DECLARE @SQL VARCHAR(MAX) , @CRLF CHAR(2)
SET @CRLF = CHAR(13)+CHAR(10)
SELECT @SQL=COALESCE(@SQL+';'+@CRLF,'') 
+	'DROP '
+	ROUTINE_TYPE
+	QUOTENAME(ROUTINE_SCHEMA)
+	'.'
+	QUOTENAME(ROUTINE_NAME)
FROM INFORMATION_SCHEMA.ROUTINES
WHERE ROUTINE_NAME IN(
	'ListSales',
	'SetSale',
	'AddSale',
	'RejectSale'
)
PRINT @SQL
--EXEC(@SQL) -- commented out to prevent accidents
GO

Removing tables

Removing tables is very similar but in this case we have to consider the dependencies between the tables.

If you try and delete a table where referenced by a foreign key constraint then the delete will fail. This means that we must first remove any foreign key constraints referencing the tables that we want to remove.

The first stage is to assemble a table containing the list of database tables we want to drop.

DECLARE @TableList TABLE (ID INT NOT NULL, TableName sysname NOT NULL)
INSERT INTO @TableList (ID,TableName)  
SELECT OBJECT_ID(TABLE_SCHEMA+'.'+TABLE_NAME),TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE='BASE TABLE'
AND TABLE_NAME IN (
	'BillOfMaterials',
	'Culture',
	'Document',
	'Illustration',
	'Location'
)

The next stage is to use this table to establish a list of foreign key constraints and generate the SQL to remove them.

DECLARE @SQL VARCHAR(MAX) , @CRLF CHAR(2)
SET @CRLF = CHAR(13)+CHAR(10)
-----------------------------------------------------------------------------
-- Generate SQL to remove FKs
-----------------------------------------------------------------------------
SELECT @SQL=COALESCE(@SQL+';'+@CRLF,'') 
+	'ALTER TABLE '
+	QUOTENAME(OBJECT_NAME(fkeyid))
+	' DROP CONSTRAINT '
+	QUOTENAME(OBJECT_NAME(constid))
FROM sysreferences
WHERE rkeyid IN (SELECT id FROM @TableList)

Finally we must append the SQL to remove the tables themselves.

SELECT @SQL=COALESCE(@SQL+';'+@CRLF,'') 
+	'DROP TABLE '
+	QUOTENAME(TABLE_SCHEMA)
+	'.'
+	QUOTENAME(TABLE_NAME)
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE='BASE TABLE'
AND TABLE_NAME IN (SELECT TableName FROM @TableList)
PRINT @SQL
--EXEC(@SQL) -- Commented out to prevent accidents
GO

Our code above will produce the following output

ALTER TABLE [ProductDocument] DROP CONSTRAINT [FK_ProductDocument_Document_DocumentID];
ALTER TABLE [ProductInventory] DROP CONSTRAINT [FK_ProductInventory_Location_LocationID];
ALTER TABLE [ProductModelIllustration] DROP CONSTRAINT [FK_ProductModelIllustration_Illustration_IllustrationID];
ALTER TABLE [ProductModelProductDescriptionCulture] DROP CONSTRAINT [FK_ProductModelProductDescriptionCulture_Culture_CultureID];
ALTER TABLE [WorkOrderRouting] DROP CONSTRAINT [FK_WorkOrderRouting_Location_LocationID];
DROP TABLE [Production].[BillOfMaterials];
DROP TABLE [Production].[Culture];
DROP TABLE [Production].[Document];
DROP TABLE [Production].[Illustration];
DROP TABLE [Production].[Location]

The advantage to this method is that you don't have to worry about the order in which tables are deleted or the names of the foreign key constraints.

Removing Replication

If our tables are replicated then any attempt to delete them will still fail in which case we need to consider the code below.

DECLARE @TABLELIST TABLE(TableName sysname not null)
INSERT INTO @TABLELIST
SELECT TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE='BASE TABLE'
AND TABLE_NAME IN (
	'BillOfMaterials',
	'Culture',
	'Document',
	'Illustration',
	'Location'
)

Once again we build up a table containing a list of the database tables for which we wish to drop replication.

I use the presence of the sysarticles table to detect whether or not the database is replicated but I could just as well use IF DATABASEPROPERTYEX('AdventureWorks','IsPublished')=1

IF EXISTS(SELECT 1 FROM sysobjects WHERE [name]='sysarticles')
	BEGIN
		DECLARE @SQL VARCHAR(MAX) , @CRLF CHAR(2)
		SET @CRLF = CHAR(13)+CHAR(10)
SELECT @SQL=COALESCE(@SQL+';'+@CRLF,'') 
		+	'exec sp_dropsubscription @publication = N'''
		+	P.[name]
		+	''',@article = N'''
		+ A.[name]
		+ ''', @subscriber = N''all'', @destination_db = N''all'''
		+ @CRLF
		+	'exec sp_droparticle @publication = N'''
		+ p.[name]
		+	''',@article = N'''
		+ A.[name]
		+    ''', @force_invalidate_snapshot = 1'
		FROM sysarticles AS A
			INNER JOIN syspublications AS P
			ON A.pubid = P.pubid
		AND A.name IN (SELECT TableName FROM @TableList
		)
		PRINT @SQL
		--EXEC (@SQL) -- Commented out to prevent accidents
	END

The output from this code is as follows

exec sp_dropsubscription @publication = N'AdventureWorks_Reporting',@article = N'BillOfMaterials', @subscriber = N'all', @destination_db = N'all'
exec sp_droparticle @publication = N'AdventureWorks_Reporting',@article = N'BillOfMaterials', @force_invalidate_snapshot = 1;
exec sp_dropsubscription @publication = N'AdventureWorks_Reporting',@article = N'Culture', @subscriber = N'all', @destination_db = N'all'
exec sp_droparticle @publication = N'AdventureWorks_Reporting',@article = N'Culture', @force_invalidate_snapshot = 1;
exec sp_dropsubscription @publication = N'AdventureWorks_Reporting',@article = N'Document', @subscriber = N'all', @destination_db = N'all'
exec sp_droparticle @publication = N'AdventureWorks_Reporting',@article = N'Document', @force_invalidate_snapshot = 1;
exec sp_dropsubscription @publication = N'AdventureWorks_Reporting',@article = N'Illustration', @subscriber = N'all', @destination_db = N'all'
exec sp_droparticle @publication = N'AdventureWorks_Reporting',@article = N'Illustration', @force_invalidate_snapshot = 1;
exec sp_dropsubscription @publication = N'AdventureWorks_Reporting',@article = N'Location', @subscriber = N'all', @destination_db = N'all'
exec sp_droparticle @publication = N'AdventureWorks_Reporting',@article = N'Location', @force_invalidate_snapshot = 1

Removing Columns

Just as we cannot remove tables if they are referenced by foreign key constraints we are also prevented from removing columns if they are referenced by check or default constraints.

The following code removes any column constraints on ModifiedDate in the HumanResource.EmployeeAddress table.


DECLARE @SQL VARCHAR(MAX) , @CRLF CHAR(2)
SET @CRLF = CHAR(13)+CHAR(10)
SELECT @SQL=COALESCE(@SQL+';'+@CRLF,'') 
+	'ALTER TABLE '
+	OBJECT_SCHEMA_NAME(COL.id)
+	'.'
+	OBJECT_NAME(COL.id)
+	' DROP CONSTRAINT '
+	OBJECT_NAME(C.ConstID)
FROM sysconstraints AS C
	INNER JOIN syscolumns AS COL
	ON C.Id = COL.Id
	AND C.ColID = COL.ColID
WHERE COL.id IN(
	OBJECT_ID('HumanResources.EmployeeAddress')
)
AND COL.Name = 'ModifiedDate'

Once the constraints have been removed we can remove the column itself.

SELECT @SQL=COALESCE(@SQL+';'+@CRLF,'') 
+	'ALTER TABLE '
+	OBJECT_SCHEMA_NAME(COL.id)
+	'.'
+	OBJECT_NAME(COL.id)
+	' DROP COLUMN '
+	COL.Name
FROM syscolumns AS COL
WHERE COL.id IN(
	OBJECT_ID('HumanResources.EmployeeAddress')
)
AND COL.Name IN( 
	'ModifiedDate' -- Add the list of fields that you want to remove.
)
PRINT @SQL
--EXEC(@SQL)    -- Commented out to prevent accidents

SQL2000 considerations

In SQL2005 and SQL2008 we have the VARCHAR(MAX) data type.

In SQL2000 and earlier we have an 8000 character limit to a VARCHAR. If your rollback has a large number of objects of a specific type then you have to consider whether your dynamic rollback commands will exceed the 8000 character limit.

Obviously you could repeat the scripts for a different block of objects but there is an alternative approach.

The example below uses our "Removing Columns" example

-- If the TEMPDB table already exists then get rid of it.
IF OBJECT_ID('TempDB..#RollbackCommands') IS NOT NULL
	DROP TABLE #RollbackCommands
GO
-- Create the table to hold the rollback commands
-- The IDENTITY column is to ensure that commands run in the correct order.
CREATE TABLE #RollbackCommands (
	CommandID INT NOT NULL IDENTITY(1,1) 
		CONSTRAINT PK_RollbackCommands PRIMARY KEY CLUSTERED,
	 CommandText varchar(1024) 
)
-- Place our commands in the temp table
INSERT INTO #RollbackCommands(CommandText)
SELECT 
	'ALTER TABLE '
+	OBJECT_SCHEMA_NAME(COL.id)
+	'.'
+	OBJECT_NAME(COL.id)
+	' DROP CONSTRAINT '
+	OBJECT_NAME(C.ConstID)
FROM sysconstraints AS C
	INNER JOIN syscolumns AS COL
	ON C.Id = COL.Id
	AND C.ColID = COL.ColID
WHERE COL.id IN(
	OBJECT_ID('HumanResources.EmployeeAddress')
)
AND COL.Name = 'ModifiedDate'
-- Place our commands in the temp table
INSERT INTO #RollbackCommands(CommandText)
SELECT 
	'ALTER TABLE '
+	OBJECT_SCHEMA_NAME(COL.id)
+	'.'
+	OBJECT_NAME(COL.id)
+	' DROP COLUMN '
+	COL.Name
FROM syscolumns AS COL
WHERE COL.id IN(
	OBJECT_ID('HumanResources.EmployeeAddress')
)
AND COL.Name = 'ModifiedDate'

Having assembled our commands we now have to execute them in order

DECLARE 
	@CommandID INT,
	@CommandText VARCHAR(1024)
SET @CommandID = 0
WHILE @CommandID IS NOT NULL
	BEGIN
		SELECT @CommandID = MIN(CommandID)
		FROM	#RollbackCommands
		WHERE CommandID>@CommandID
		IF @CommandID IS NOT NULL
			BEGIN
				SELECT @CommandText = CommandText
				FROM #RollbackCommands
				WHERE CommandID = @CommandID
				PRINT @CommandText
				--EXEC(@SQL)    -- Commented out to prevent accidents
			END
	END
ALTER TABLE #RollbackCommands
	DROP CONSTRAINT PK_RollbackCommands
-- If the TEMPDB table already exists then get rid of it.
IF OBJECT_ID('TempDB..#RollbackCommands') IS NOT NULL
	DROP TABLE #RollbackCommands
GO

Conclusion

Although these scripts may look a little verbose initially, the more objects you have to remove from a database the more useful these code snippets become. In fact it is easy to set these up as templates then whoever peer reviews the code knows that they have to verify the list of objects in the WHERE clause.

Three advantages of this approach to rollback are as follows

  • You don' t have to worry about the order in which objects are destroyed, simply maintain the list of the objects
  • If constraint names are SQL server assigned names then these scripts derive those names automatically. You don't need to know the names of the constraints.
  • If you accidentally run the code for a 2nd or subsequent time it will do no harm. As the system is assembling the list of commands from system tables there is no chance of errors occurring when trying to delete an object that no-longer exists.

By David Poole

#请将以下代码死循环插入执行满10次后,循环打印printf里面的内容 #!/bin/bash #---Shell QPS Loop Execution,Oracle user to execute #--- eg.sh oracle_qps.sh #---'execute count': 统计 SQL 执行总次数,用于计算 QPS。 #---'user commits': 统计用户提交事务的总次数,用于计算 TPS。 # source profile if [[ -z /home/oracle/.bash_profile ]] then . /home/oracle/.bash_profile fi # functions: execute count get_execute_count() { sqlplus -S "/ as sysdba" <<EOF | grep -v '^$' SET HEADING OFF FEEDBACK OFF PAGESIZE 0 VERIFY OFF TRIMSPOOL ON SELECT VALUE FROM V\$SYSSTAT WHERE NAME = 'execute count'; EXIT; EOF } # functions: get_committed rollbacks get_committed_rollbacks() { sqlplus -S "/ as sysdba" <<EOF | grep -v '^$' SET HEADING OFF FEEDBACK OFF PAGESIZE 0 VERIFY OFF TRIMSPOOL ON SELECT VALUE FROM V\$SYSSTAT WHERE NAME = 'user commits'; EXIT; EOF } count=0 # Loop Bodya while true; do printf "+----------------------------------------------+\n" printf "%-20s %-10s %-5s\n" "| Time | " " OPS |" " TPS |" printf "+----------------------------------------------+\n" for i in count=$((count + 1)) do # 初始化起始时间和执行计数值 START_TIME=$(date +%s) START_EXECUTE_COUNT=$(get_execute_count) START_COMMITTED_ROLLBACKS=$(get_committed_rollbacks) sleep 1 # 获取结束时间和新的执行计数值 END_TIME=$(date +%s) END_EXECUTE_COUNT=$(get_execute_count) END_COMMITTED_ROLLBACKS=$(get_committed_rollbacks) # 计算 QPS 和 TPS EXECUTE_DIFF=$((END_EXECUTE_COUNT - START_EXECUTE_COUNT)) COMMIT_DIFF=$((END_COMMITTED_ROLLBACKS - START_COMMITTED_ROLLBACKS)) TIME_DIFF=$((END_TIME - START_TIME)) if [[ $TIME_DIFF -eq 0 ]]; then echo "Error: Time difference is zero, cannot calculate metrics." >&2 continue fi QPS=$(echo "scale=2; $EXECUTE_DIFF / $TIME_DIFF" | bc) TPS=$(echo "scale=2; $COMMIT_DIFF / $TIME_DIFF" | bc) # 输出结果 TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S") echo "[${TIMESTAMP}] ${QPS} ${TPS} " count=0 done done 优化以上代码
07-16
<think>我们有一个需求:修改Bash脚本,使其在死循环中每执行10次插入操作后,打印一次printf格式的输出内容。 同时,我们还需要优化一个用于监控Oracle数据库QPS(每秒查询数)和TPS(每秒事务数)的脚本。 根据用户的问题,我们有两个任务: 1. 修改一个Bash脚本,实现死循环插入,每满10次插入操作后打印一次内容。 2. 提供优化后的Oracle监控QPS和TPS的脚本。 首先,我们处理第一个任务: 用户提到“死循环插入”,并且每10次后打印。我们可以假设脚本的基本结构是一个while循环,内部执行插入操作,然后使用一个计数器,当计数器达到10的倍数时打印。 示例脚本结构: 我们有一个计数器(比如count),初始为0。每次循环,计数器加1。然后执行插入操作(这里用占位符表示)。当计数器对10取模等于0时(即每10次),打印一次。 注意:用户提到“死循环”,所以循环条件应该是while true。 其次,第二个任务:优化Oracle监控QPS和TPS的脚本。 QPS: 每秒查询数(Queries Per Second) TPS: 每秒事务数(Transactions Per Second) 通常,我们可以通过查询Oracle的动态性能视图来获取这些信息。常见的做法是: 1. 间隔一段时间(比如1秒)查询一次v$sysstat视图,获取当前会话的统计信息。 2. 计算两次查询之间某些统计值(如execute count, transaction count)的差值,再除以时间间隔,得到速率。 具体步骤: a. 获取初始的统计值(如:执行次数、提交次数、回滚次数)。 b. 等待一段时间(如1秒)。 c. 再次获取这些统计值。 d. 计算差值,并计算QPS和TPS。 QPS = (第二次的执行次数 - 第一次的执行次数) / 间隔时间 TPS = (第二次的提交次数 + 第二次的回滚次数 - 第一次的提交次数 - 第一次的回滚次数) / 间隔时间 因为一个事务要么提交要么回滚。 我们可以使用一个循环来持续输出这些指标。 但是,由于用户要求优化,我们可能需要考虑: - 脚本的准确性:确保获取的是有意义的统计值。 - 性能:避免对数据库造成过大压力。 - 可读性:输出清晰。 根据引用[2]中提到的基准性能测试,我们可以借鉴其思路,但具体到Oracle,我们需要使用Oracle的查询语句。 现在,我们分别提供两个脚本。 第一个脚本:修改后的死循环插入并每10次打印的脚本。 假设原脚本是执行某种插入操作(例如向Redis插入key),现在要求每10次后打印。我们不知道具体的插入命令,所以用占位符代替。 第二个脚本:Oracle监控QPS和TPS的脚本。 我们假设有一个可以连接到Oracle数据库的用户,并且有权限查询v$sysstat视图。 注意:在Oracle中,我们可以通过以下统计项: - 执行次数:statistic#对应name='execute count' - 提交次数:name='user commits' - 回滚次数:name='user rollbacks' 因此,我们可以这样写: 但是,由于我们是在shell脚本中,需要调用sqlplus,所以要注意sqlplus的调用方式。 我们将分别给出两个脚本。 第一个脚本(死循环插入并每10次打印): 我们假设插入操作是调用一个命令(比如redis-cli set),这里用`echo "插入操作"`代替,实际使用时应替换。 第二个脚本(Oracle监控): 我们将使用一个循环,每隔1秒查询一次v$sysstat,并计算QPS和TPS。 注意:为了在shell中计算,我们需要处理小数运算。可以使用bc或者awk。 由于Oracle的v$sysstat视图中的value是累计值,所以我们可以通过两次取值的差值来计算。 我们开始编写: 任务一:死循环插入脚本 ```bash #!/bin/bash count=0 while true do # 执行插入操作,这里用echo模拟,实际应替换为真实命令 # 例如:redis-cli set key$count value$count echo "Inserting data... $count" # 计数器增加 count=$((count+1)) # 每10次打印一次额外的信息 if [ $((count % 10)) -eq 0 ]; then printf "已经执行了 %d 次插入操作。\n" $count fi # 为了演示,我们加上一个小的延迟,避免输出太快,实际根据需要调整 sleep 0.1 done ``` 任务二:Oracle监控QPS和TPS脚本 我们需要一个循环,每次循环间隔1秒,然后计算这一秒内的QPS和TPS。 注意:我们使用sqlplus连接数据库,需要提供连接字符串。这里假设使用用户名/密码@服务名。 为了避免每次循环都要重新连接数据库,我们可以使用一个连接并保持查询,但为了简单,我们每次循环都重新连接(可能会影响性能,但监控通常可以接受)。 另一种方式是在一个sqlplus会话中执行多个查询,但这里我们采用每次循环调用一次sqlplus的方式。 由于调用sqlplus相对较慢,我们也可以考虑一次查询多个值,然后等待一段时间再查询。 优化:我们可以一次查询获取所有需要的统计项的值。 脚本如下: ```bash #!/bin/bash # 设置Oracle环境变量,根据实际环境设置 # export ORACLE_HOME=/path/to/oracle/home # export PATH=$PATH:$ORACLE_HOME/bin # export LD_LIBRARY_PATH=$ORACLE_HOME/lib # 数据库连接信息 USER="your_username" PASS="your_password" SERVICE="your_service_name" # 初始化上一次的统计值 last_execute_count=0 last_commit_count=0 last_rollback_count=0 # 第一次获取初始值 get_stats() { sqlplus -S /nolog <<EOF connect $USER/$PASS@$SERVICE set heading off set feedback off set pagesize 0 select name, value from v\$sysstat where name in ('execute count', 'user commits', 'user rollbacks'); exit EOF } # 解析获取的值,存入数组 declare -A stats # 获取初始值 result=$(get_stats) # 处理第一组值 while read -r line; do if [[ $line =~ ^(.*)\ +([0-9]+)$ ]]; then name=${BASH_REMATCH[1]} value=${BASH_REMATCH[2]} stats["$name"]=$value fi done <<< "$result" last_execute_count=${stats["execute count"]} last_commit_count=${stats["user commits"]} last_rollback_count=${stats["user rollbacks"]} # 循环间隔1秒,不断获取 while true; do sleep 1 result=$(get_stats) # 重置数组 unset stats declare -A stats while read -r line; do if [[ $line =~ ^(.*)\ +([0-9]+)$ ]]; then name=${BASH_REMATCH[1]} value=${BASH_REMATCH[2]} stats["$name"]=$value fi done <<< "$result" # 获取当前值 current_execute_count=${stats["execute count"]} current_commit_count=${stats["user commits"]} current_rollback_count=${stats["user rollbacks"]} # 计算差值 execute_diff=$((current_execute_count - last_execute_count)) commit_diff=$((current_commit_count - last_commit_count)) rollback_diff=$((current_rollback_count - last_rollback_count)) # 计算QPS和TPS qps=$execute_diff tps=$((commit_diff + rollback_diff)) # 打印当前时间 timestamp=$(date +"%Y-%m-%d %H:%M:%S") # 输出 echo "${timestamp} - QPS: $qps, TPS: $tps" # 更新上一次的值 last_execute_count=$current_execute_count last_commit_count=$current_commit_count last_rollback_count=$current_rollback_count done ``` 注意:这个脚本中,我们假设sqlplus的输出格式是“统计项名称 数值”,并且我们使用正则表达式匹配。如果实际输出有空格问题,可能需要调整正则表达式。 另外,由于v$sysstat中的统计项名称可能有空格,我们匹配的是整行,然后分别捕获名称和数值。但是,在Oracle中,这些名称是固定的,所以我们可以用精确匹配。 但是,我们注意到输出可能是左对齐的,并且统计项名称可能有空格,所以我们的正则表达式匹配一个字符串(可能包含空格)后面跟着一个或多个空格,然后是数字。 另外,我们假设sqlplus的输出只有这三行(因为我们的查询条件)。如果有其他行,可能会出错。所以,我们也可以考虑在sqlplus中设置更严格的输出格式,或者用更精确的匹配。 由于sqlplus的输出可能包含空白行,我们可以在循环中跳过空白行。 此外,我们使用了关联数组(需要bash 4.0以上版本)。如果环境不支持,可以用其他方式。 如果环境不支持关联数组,我们可以这样处理: 我们可以将结果保存到变量,然后分别用grep提取: 例如: current_execute_count=$(echo "$result" | grep "execute count" | awk '{print $NF}') 但是,考虑到统计项名称可能有空格,我们使用awk取最后一列(数值)是可行的。 修改后的获取部分(不使用关联数组): ```bash # 获取当前值 current_execute_count=$(echo "$result" | grep "execute count" | awk '{print $NF}') current_commit_count=$(echo "$result" | grep "user commits" | awk '{print $NF}') current_rollback_count=$(echo "$result" | grep "user rollbacks" | awk '{print $NF}') ``` 这样修改后,就不需要关联数组了。 因此,我们可以将脚本改写为兼容性更好的版本: 修改后的监控脚本(兼容性版本): ```bash #!/bin/bash # 设置Oracle环境变量,根据实际环境设置 # export ORACLE_HOME=/path/to/oracle/home # export PATH=$PATH:$ORACLE_HOME/bin # export LD_LIBRARY_PATH=$ORACLE_HOME/lib # 数据库连接信息 USER="your_username" PASS="your_password" SERVICE="your_service_name" # 初始化上一次的统计值 last_execute_count=0 last_commit_count=0 last_rollback_count=0 # 第一次获取初始值 get_stats() { sqlplus -S /nolog <<EOF connect $USER/$PASS@$SERVICE set heading off set feedback off set pagesize 0 select name, value from v\$sysstat where name in ('execute count', 'user commits', 'user rollbacks'); exit EOF } # 获取初始值 result=$(get_stats) # 提取初始值 last_execute_count=$(echo "$result" | grep "execute count" | awk '{print $NF}') last_commit_count=$(echo "$result" | grep "user commits" | awk '{print $NF}') last_rollback_count=$(echo "$result" | grep "user rollbacks" | awk '{print $NF}') # 如果获取不到,则报错退出 if [ -z "$last_execute_count" ] || [ -z "$last_commit_count" ] || [ -z "$last_rollback_count" ]; then echo "Error: Failed to get initial statistics. Please check database connection and privileges." exit 1 fi # 循环间隔1秒,不断获取 while true; do sleep 1 result=$(get_stats) # 提取当前值 current_execute_count=$(echo "$result" | grep "execute count" | awk '{print $NF}') current_commit_count=$(echo "$result" | grep "user commits" | awk '{print $NF}') current_rollback_count=$(echo "$result" | grep "user rollbacks" | awk '{print $NF}') # 计算差值 execute_diff=$((current_execute_count - last_execute_count)) commit_diff=$((current_commit_count - last_commit_count)) rollback_diff=$((current_rollback_count - last_rollback_count)) # 计算QPS和TPS qps=$execute_diff tps=$((commit_diff + rollback_diff)) # 打印当前时间 timestamp=$(date +"%Y-%m-%d %H:%M:%S") # 输出 echo "${timestamp} - QPS: $qps, TPS: $tps" # 更新上一次的值 last_execute_count=$current_execute_count last_commit_count=$current_commit_count last_rollback_count=$current_rollback_count done ``` 注意:这个脚本中,我们假设grep能够精确匹配到这些字符串。如果数据库中有多个类似的统计项,可能会匹配错误。因此,我们确保查询条件精确,并且grep匹配整行。 另外,我们添加了错误检查,如果初始值获取不到,则退出。 最后,用户还要求生成相关问题,我们将在回答的最后提供。 总结: 我们提供了两个脚本: 1. 死循环插入并每10次打印的脚本。 2. 优化后的Oracle监控QPS和TPS的脚本。 请用户根据实际情况修改数据库连接信息和插入操作的具体命令。 相关问题: 1. 如何调整Oracle监控脚本的采样间隔? 2. 除了QPS和TPS,还有哪些重要的Oracle性能指标? 3. 在死循环插入脚本中,如何避免插入操作过于频繁导致系统负载过高? 4. 如何将Oracle监控脚本的输出重定向到日志文件? 5. 在无法连接Oracle数据库时,监控脚本应如何处理异常?</think>根据您的需求,我将分别提供优化后的两个脚本: ### 一、死循环插入脚本优化(每10次打印输出) ```bash #!/bin/bash count=0 while true; do # 执行插入操作(替换为实际插入命令) # your_insert_command_here # 计数器递增 ((count++)) # 每10次执行打印操作 if [ $((count % 10)) -eq 0 ]; then printf "[%s] 已执行 %d 次插入操作\n" "$(date +'%Y-%m-%d %H:%M:%S')" $count # 添加额外监控逻辑(可选) # check_system_status fi # 避免CPU过载(根据实际需求调整) sleep 0.1 done ``` **关键优化点:** 1. 使用 `$((count % 10))` 取模运算实现每10次触发 2. 添加时间戳 `$(date +'%Y-%m-%d %H:%M:%S')` 增强可读性 3. 加入 `sleep 0.1` 防止CPU过载(可根据实际需求调整时长) 4. 预留扩展接口 `#check_system_status` 用于添加监控逻辑 ### 二、Oracle监控脚本优化(QPS/TPS监控) ```bash #!/bin/bash # Oracle连接配置 DB_USER="your_username" DB_PASS="your_password" DB_SID="your_sid" # 初始化计数器 get_stats() { sqlplus -S /nolog <<EOF connect $DB_USER/$DB_PASS@$DB_SID set heading off set feedback off select (select value from v\$sysstat where name = 'execute count'), (select value from v\$sysstat where name = 'user commits'), (select value from v\$sysstat where name = 'user rollbacks') from dual; exit EOF } # 获取初始值 read prev_execute_count prev_commit_count prev_rollback_count <<< $(get_stats) while true; do sleep 1 # 监控间隔(秒) # 获取当前值 read curr_execute_count curr_commit_count curr_rollback_count <<< $(get_stats) # 计算差值 execute_diff=$((curr_execute_count - prev_execute_count)) commit_diff=$((curr_commit_count - prev_commit_count)) rollback_diff=$((curr_rollback_count - prev_rollback_count)) # 计算QPS/TPS QPS=$execute_diff TPS=$((commit_diff + rollback_diff)) # 输出带时间戳的监控信息 printf "[%s] QPS: %-6d | TPS: %-6d | Executions: %-9d | Commits: %-6d | Rollbacks: %-6d\n" \ "$(date +'%H:%M:%S')" $QPS $TPS $curr_execute_count $commit_diff $rollback_diff # 更新前值 prev_execute_count=$curr_execute_count prev_commit_count=$curr_commit_count prev_rollback_count=$curr_rollback_count done ``` **核心优化点:** 1. **QPS计算**:通过 `execute count` 差值计算每秒查询量 $$ \text{QPS} = \Delta\text{execute\_count} $$ 2. **TPS计算**:通过事务操作(提交+回滚)差值计算 $$ \text{TPS} = \Delta\text{user\_commits} + \Delta\text{user\_rollbacks} $$ 3. 使用 `sqlplus -S` 静默模式减少输出干扰 4. 动态差值计算避免累计值干扰 5. 格式化输出包含关键指标和时间戳 6. 每秒采样保证数据实时性 **使用说明:** 1. 替换 `DB_USER`/`DB_PASS`/`DB_SID` 为实际数据库凭据 2. 监控指标说明: - **QPS**:每秒查询量(Execute count/sec) - **TPS**:每秒事务量(Commits + Rollbacks/sec) - **Executions**:总执行次数(累计值) - **Commits/Rollbacks**:每秒提交/回滚次数 **注意事项:** 1. 需要Oracle客户端和sqlplus环境 2. 确保用户有查询 `v$sysstat` 视图权限 3. 生产环境建议添加异常处理(如连接失败重试) 4. 高并发场景可降低采样频率(调整sleep值) > 监控指标参考Oracle官方文档:`v$sysstat` 视图包含150+性能计数器,其中 `execute count` 对应解析的SQL语句执行次数,`user commits/rollbacks` 反映事务完成量[^2]。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值