druid查询源码追踪

待整理和完善
org.apache.druid.cli.Main --> CliBroker getModules()--> BrokerQueryResource.getQueryTargets()
 --> 1.List<LocatedSegmentDescriptor> ServerViewUtil.getTargetLocations(BrokerServerView,datasource,interval,...) 获取segmentLocation,   2.ResourceIOReaderWriter.ok()读取segment

LocatedSegmentDescriptor信息包括:
	segmentMetadata:segment元信息(interval,version,partition,size)
	serverMetadata:实时节点和历史节点地址等。

读取segment
-->Response.ok(newOutputWriter(null, null, false).writeValueAsString(object), contentType) -->


Jetty server(Jetty+Jersey+guice组合)
Broker初始化:
1.CliBroker绑定了 BrokerServerView,CachingClusteredClient,ClientQuerySegmentWalker,TimelineServerView,BrokerQueryResource等
查询入口doPost
QueryResource.doPost()-->
1.根据请求创建对应Query对象,
2.执行查询 final QueryLifecycle.QueryResponse queryResponse = queryLifecycle.execute();

执行查询,返回sequence结果
final Sequence res = QueryPlus.wrap(baseQuery).withIdentity(authenticationResult.getIdentity()).run(texasRanger, responseContext); -->  QueryRunner.run()
创建QueryRunner,执行QueryRunner.run();
说明:
   1.ClientQuerySegmentWalker texasRanger:创建QueryRunner
   2.ResponseContext responseContext:
	   The context for storing and passing data between chains of {
   
   @link org.apache.druid.query.QueryRunner}s.
	   The context is also transferred between Druid nodes with all the data it contains.
   3. QueryPlus 包含Query,QueryMetries,Identity  
   4. ClientQuerySegmentWalker创建QueryRunner
    一种根据Query和interval创建对应的QueryRunner
	一种根据SegmentDescriptor创建对应的QueryRunner

ClientQuerySegmentWalker:根据查询中的intervals或segments参数来确定调用getQueryRunnerForIntervals还是getQueryRunnerForSegments
public <T> QueryRunner<T> getQueryRunnerForIntervals(Query<T> query, Iterable<Interval> intervals)
  {
   
   
    return makeRunner(query, baseClient.getQueryRunnerForIntervals(query, intervals));
  }
  
makeRunner:返回ResultLevelCachingQueryRunner,ResultLevelCachingQueryRunner装饰FluentQueryRunner,FluentQueryRunner装饰SetAndVerifyContextQueryRunner,SetAndVerifyContextQueryRunner装饰了RetryQueryRunner,RetryQueryRunner装饰baseRunner

getQueryRunnerForIntervals():返回匿名baseRunner,内部run调用 CachingClusteredClient.run --》CachingClusteredClient内部类SpecificQueryRunnable.run

SpecificQueryRunnable.run: 
Sequence<T> run(final UnaryOperator<TimelineLookup<String, ServerSelector>> timelineConverter)
    {
   
   
	//根据datasource获取timeline
      @Nullable
      TimelineLookup<String, ServerSelector> timeline = serverView.getTimeline(query.getDataSource());
      if (timeline == null) {
   
   
        return Sequences.empty();
      }
      timeline = timelineConverter.apply(timeline);
      if (uncoveredIntervalsLimit > 0) {
   
   
        computeUncoveredIntervals(timeline);
      }
//根据timeline获取ServerSelector与对应SegmentDescriptor
      final Set<ServerToSegment> segments = computeSegmentsToQuery(timeline);
	  //根据query获取缓存key
      @Nullable
      final byte[] queryCacheKey = computeQueryCacheKey();
      if (query.getContext().get(QueryResource.HEADER_IF_NONE_MATCH) != null) {
   
   
        @Nullable
        final String prevEtag = (String) query.getContext().get(QueryResource.HEADER_IF_NONE_MATCH);
        @Nullable
        final String currentEtag = computeCurrentEtag(segments, queryCacheKey);
        if (currentEtag != null && currentEtag.equals(prevEtag)) {
   
   
          return Sequences.empty();
        }
      }
//根据缓存key获取对应的缓存结果
      final List<Pair<Interval, byte[]>> alreadyCachedResults = pruneSegmentsWithCachedResults(queryCacheKey, segments);
	  //获取Server以及所拥有的 SegmentDescriptor集合
      final SortedMap<DruidServer, List<SegmentDescriptor>> segmentsByServer = groupSegmentsByServer(segments);
      return new LazySequence<>(<
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值