azkaban 多执行端部署验证

azkaban web 端添加配置 azkaban.properties:

azkaban web 端添加配置  azkaban.properties	
azkaban.use.multiple.executors=true
azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1
azkaban.executorselector.comparator.Memory=1
azkaban.executorselector.comparator.LastDispatched=1
azkaban.executorselector.comparator.CpuUsage=1
azkaban.activeexecutor.refresh.milisecinterval=1000
azkaban.activeexecutor.refresh.flowinterval=1
azkaban.executorinfo.refresh.maxThreads=1
Servers	Executor servers:
192.168.11.164
192.168.11.142
192.168.11.105

Azkaban web:
192.168.11.105

Mysql:
192.168.11.86
use azkaban;	
 	 

NO.	Operation	Tables & SQL	Result(my test result sample)
1	After R&D config the azkaban web/executors and restart,executors should be automatically registered successfully;	Check the registration results:
select * from  executors;	
| id | host                              | port  | active |
+----+-----------------------------------+-------+--------+
| 62 | THA-168-11-164.h.chinabank.com.cn | 12321 |      1 |
| 64 | THA-168-11-142.h.chinabank.com.cn | 12321 |      1 |
| 66 | THA-168-11-105.h.chinabank.com.cn | 12321 |      1 |

2	Execute the old Azkaban job, its should be randomly assigned to one of the executors ;	
Check this job's executor id:

SELECT executor_id FROM `execution_flows` where exec_id= 	executor_id 
+-------------+
|          64 |
3	Create data ingestion job,it should be randomly assigned to one of the executors ;	Check the executor id :
SELECT executor_id FROM `execution_flows` where exec_id=	
| executor_id |
+-------------+
|          66 |
4	Execute multiple Azkaban jobs ,they assigned to the executor randomly;	Check the executors id:
SELECT executor_id FROM `execution_flows` where exec_id=;	
| executor_id |
+-------------+
|          64 |
|          66 |
|          64 |
|          66 |
|          64 |
|          62 |
|          64 |

5	The same Azkaban job execute on the different executor servers(Run the job many times manually);	Check the executors id:

SELECT project_id, executor_id FROM `execution_flows` ORDER BY exec_id desc LIMIT 2;	
| project_id | executor_id |
+------------+-------------+
|       2562 |          66 |
|       2562 |          64 |
6	When Azkaban job is running, stop the server it is using,this job's execution result should be failed, re-executed it manually should be run on another executor:	Check the executors id:

SELECT project_id, exec_id,executor_id FROM `execution_flows` ORDER BY exec_id desc limit 5;	
| project_id | executor_id |
+------------+-------------+
|       2581 |          64 |
|       2581 |          66 |
7	Multiple Azkaban jobs are executed at the same time,they should be randomly assigned to the executor;	Check the executors id:

SELECT project_id, exec_id,executor_id FROM `execution_flows` ORDER BY exec_id desc limit 55;
(Since we reconfigured 105 server, its id changed from 66 to 68)	
| project_id | exec_id | executor_id |
+------------+---------+-------------+
|       2624 |   88642 |          62 |
|       2627 |   88641 |          62 |
|       2619 |   88640 |          62 |
|       2353 |   88639 |          64 |


评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值