近日生产发生死锁,日志和表索引如下。
分析是
因为索引的原因,在循环对数据进行Update的时候,因为查询的结果集颗粒度很大,造成多个线程因为索引INX_ACCOUNT_ID产生了行级锁。
解决办法是,缩小查询结果集的颗粒度,就是缩小查询的结果范围,在循环Update的时候不会造成资源竞争。
循环如下
//新的循环
List<Map<String,Object>> accountMap=jsperManagerService.selectJsperDataUnProcessMoreKey();
logger.info("STORM SPOUT UNACCOUNTID LIST SIZE IS " + accountMap.size());
// TODO 这里Spout业务
if(CollectionUtils.isNotEmpty(accountMap)){
logger.info("STORM SPOUT BIGLOOP START " + System.currentTimeMillis());
for(Map<String,Object> mapAccountId:accountMap){
if(mapAccountId.get("ACCOUNT_ID")!=null){
String accountId = "";
String number = "";
accountId = String.valueOf(mapAccountId.get("ACCOUNT_ID"));
number = String.valueOf(mapAccountId.get("NUMBER"));
List<Map<String,Object>> list = jsperManagerService.selectJsperInterDataMoreKey(accountId,number);//获取接口信息
logger.info("STORM SPOUT LITTLELOOP START " + System.currentTimeMillis());
for(Map<String,Object> map:list){
if(map != null){
if(updateJsperStatus(map,true)){
outputCollector.emit(new Values(map));
}
}
}
logger.info("STORM SPOUT LITTLELOOP END " + System.currentTimeMillis());
}
}
logger.info("STORM SPOUT BIGLOOP END " + System.currentTimeMillis());
}
//原有循环
//List<Map<String,Object>> unaccountId=jsperManagerService.selectJsperDataUnProcess();
// logger.info("STORM SPOUT UNACCOUNTID LIST SIZE IS " + unaccountId.size());
// // TODO 这里Spout业务
// if(CollectionUtils.isNotEmpty(unaccountId)){
// logger.info("STORM SPOUT BIGLOOP START " + System.currentTimeMillis());
// for(Map<String,Object> mapAccountId:unaccountId){
// if(mapAccountId.get("ACCOUNT_ID")!=null){
// List<Map<String,Object>> list = jsperManagerService.selectJsperInterData(mapAccountId.get("ACCOUNT_ID").toString());//获取接口信息
// logger.info("STORM SPOUT LITTLELOOP START " + System.currentTimeMillis());
// for(Map<String,Object> map:list){
// if(map != null){
// //JpInterCallerDto dto = (JpInterCallerDto) ConversionUtil.map2po(map, JpInterCallerDto.class);
// if(updateJsperStatus(map,true)){
// outputCollector.emit(new Values(map));
// }
// }
// }
// logger.info("STORM SPOUT LITTLELOOP END " + System.currentTimeMillis());
// }
// }
// logger.info("STORM SPOUT BIGLOOP END " + System.currentTimeMillis());
// }
索引
MariaDB [jliot]> show index in JP_INTER_CALLER;
+-----------------+------------+----------------------+--------------+-------------+-----------+-------------+----------+--------+------+----------- -+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------+------------+----------------------+--------------+-------------+-----------+-------------+----------+--------+------+----------- -+---------+---------------+
| jp_inter_caller | 1 | JP_CALLER_INX | 1 | CREATE_TIME | A | 234671 | NULL | NULL | | BTREE | | |
| jp_inter_caller | 1 | JP_CALLER_CUSTID_INX | 1 | CUST_NUMBER | A | 1173359 | NULL | NULL | YES | BTREE | | |
| jp_inter_caller | 1 | INX_ACCOUNT_ID | 1 | USE_TAG | A | 14 | NULL | NULL | YES | BTREE | | |
| jp_inter_caller | 1 | INX_ACCOUNT_ID | 2 | ACCOUNT_ID | A | 96 | NULL | NULL | YES | BTREE | | |
| jp_inter_caller | 1 | JP_CALLER_PARAM3_INX | 1 | PARAM3 | A | 28 | NULL | NULL | YES | BTREE | | |
+-----------------+------------+----------------------+--------------+-------------+-----------+-------------+----------+--------+------+----------- -+---------+---------------+
5 rows in se