Region Server之间coprocessor相互调用

最新推荐文章于 2021-09-17 14:16:08 发布

原创最新推荐文章于 2021-09-17 14:16:08 发布 · 297 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#HBase #图计算

HBase 专栏收录该内容

2 篇文章

订阅专栏

本文介绍了一种使用HBase Coprocessor机制减少应用与HBase集群间网络交互次数的方法，通过在RegionServer上执行多步查询并整合结果，有效降低调用次数，提升查询效率。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

如下图所示，是一个利用HBase Coprocessor机制来减少应用和HBase集群间多步串行查询网络交互次数的方法。

图的大体流程是，应用将查询的rowkey等参数传递到Region Server，然后Region Server先做自己所在Region的查询，将本机查询的结果作为下次查询的Rowkey，并将在本Region的Rowkey通过RPC请求其他Region Server，多步查询后将结果一次返回给应用，从而减少应用和HBase集群间的调用次数。

1. 编写coprocessor

步骤参照实际动手编写HBase Coprocessor

2. coprocessor的核心查询代码片段如下

@Override
    public void getAdjacentVertices(RpcController controller, GraphTravesalProtocol.TraversalRequest request, RpcCallback<GraphTravesalProtocol.TraversalResponse> done) {

        /**
         * 1. If call by local region server itself, just return.
         */
        long reqRegionId = request.getRegionId();
        if(reqRegionId == env.getRegionInfo().getRegionId()){
            GraphTravesalProtocol.TraversalResponse response = localRegionServerRPCCallProcess();
            done.run(response);
            return;
        }

        /**
         * 2. Iterate execute traversal on local region server
         *    and find out vertex ids which not in local region server.
         */
        List<GraphTravesalProtocol.TravesalStep> travesalSteps = new ArrayList<GraphTravesalProtocol.TravesalStep>(request.getTravesalStepsList());
        List<GraphTravesalProtocol.TravesalStep> travesalStepsNotInLocalRegionServer = new ArrayList<GraphTravesalProtocol.TravesalStep>();
        for(GraphTravesalProtocol.TravesalStep travesalStep : travesalSteps){
            stepExecution(travesalSteps, travesalStep, travesalStepsNotInLocalRegionServer);
        }

        /**
         * 3. RPC call other region server to execute traversal
         *    which vertex ids not in local region server.
         */
        Map<byte[], GraphTravesalProtocol.TraversalResponse> responseMap = null;
        try {
            responseMap = GraphTravesalClient.getInstance().remoteTraversal(travesalStepsNotInLocalRegionServer, env.getRegionInfo().getRegionId());
        } catch (Throwable throwable) {
            log.error(throwable.getStackTrace());
        }

        /**
         * 4. Generate Response
         */
        GraphTravesalProtocol.TraversalResponse.Builder builder = GraphTravesalProtocol.TraversalResponse.newBuilder();
        /** 4.1 Merge RPC call response from other region server. */
        Iterator iterator = responseMap.keySet().iterator();
        while (iterator.hasNext()){
            byte[] key = (byte[]) iterator.next();
            GraphTravesalProtocol.TraversalResponse response = responseMap.get(key);
            builder.addAllVertices(response.getVerticesList());
        }

        /** 4.2 Local region vertex ids merge */
        GraphTravesalProtocol.TravesalStep finalStep = travesalSteps.get(travesalSteps.size() - 1 );
        builder.addAllVertices(finalStep.getAdjacentVerticesList());

        /**
         * 5. Return response
         */
        GraphTravesalProtocol.TraversalResponse response = builder.build();
        done.run(response);

    }

更多详情请关注：娃娃学软件