elastic-job简单的分片

Elastic-Job分片

AbstractElasticJob在执行job的时候会获取分片内容JobExecutionMultipleShardingContext，通过shardingService.shardingIfNecessary()进行分片。分片默认得分片策略是均分算法AverageAllocationJobShardingStrategy.sharding()，获取本机的分片序列号shardingService.getLocalHostShardingItems()，并把分片序列号进行set。我们在写具体的业务job，实现process方法时候，可以通过shardingContext获取分片序列号shardingItems。

例如：

投标分10片，那么每台服务器都会获取到对应的分片序列号。当一台服务器那么分片序列号是0,1,2,3,4,5,6,7,8,9，当两台服务器的时候分片序列号是服务器A-0,1,2,3,4服务器B-5,6,7,8,9，实现可以根据查询业务数据进行进行%10取余，如果取到的余在获取的分片序列号内进行投标处理，如果不在则不处理。投标考虑标id和账户id都进行%10是否在满足分片进行具体的投标处理。

序列号对应参数配置shardingItemParameters

部分代码片度如下：

均分策略AverageAllocationJobShardingStrategy

附测试代码：

import java.util.ArrayList;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;


import com.alibaba.fastjson.JSONObject;
import com.beust.jcommander.internal.Lists;
import com.dangdang.ddframe.job.internal.sharding.strategy.JobShardingStrategyOption;
import com.google.common.collect.Maps;


public class Testss {
public static void main(String[] args) {


List<String> serversList = Lists.newArrayList(2);
serversList.add("ip1");
serversList.add("ip2");
int shardingTotalCount = 10;

Map<Integer, String> shardingItemParameters  = Maps.newHashMapWithExpectedSize(0);
JobShardingStrategyOption option = new JobShardingStrategyOption("autoLoanBid", shardingTotalCount, shardingItemParameters);
Map<String, List<Integer>> map = sharding(serversList, option);
System.out.println(JSONObject.toJSONString(map));

}


public static Map<String, List<Integer>> sharding(final List<String> serversList, final JobShardingStrategyOption option) {
if (serversList.isEmpty()) {
return Collections.emptyMap();
}
Map<String, List<Integer>> result = shardingAliquot(serversList, option.getShardingTotalCount());
addAliquant(serversList, option.getShardingTotalCount(), result);
return result;
}


private static Map<String, List<Integer>> shardingAliquot(final List<String> serversList, final int shardingTotalCount) {
Map<String, List<Integer>> result = new LinkedHashMap<>(serversList.size());
int itemCountPerSharding = shardingTotalCount / serversList.size();
int count = 0;
for (String each : serversList) {
List<Integer> shardingItems = new ArrayList<>(itemCountPerSharding + 1);
for (int i = count * itemCountPerSharding; i < (count + 1) * itemCountPerSharding; i++) {
shardingItems.add(i);
}
result.put(each, shardingItems);
count++;
}
return result;
}


private static void addAliquant(final List<String> serversList, final int shardingTotalCount, final Map<String, List<Integer>> shardingResult) {
int aliquant = shardingTotalCount % serversList.size();
int count = 0;
for (Entry<String, List<Integer>> entry : shardingResult.entrySet()) {
if (count < aliquant) {
entry.getValue().add(shardingTotalCount / serversList.size() * serversList.size() + count);
}
count++;
}
}


}